Mr Davis was seen by journalists leaning in towards the Shadow Home Secretary for what people assumed was a hug or a kiss after the crucial Commons vote.
But the Shadow Home Secretary recoiled, allegedly telling him to “f*** off” before he “walked off laughing”.
However, it has now emerged the Tory frontbencher leant in to whisper in her ear “thanks for your vote”.
The bizarre exchange happened after Ms Abbott voted in favour of the Tory Brexit Bill in its final Commons reading earlier this week.
People assumed Mr Davis leant in for a kiss or a hug
Despite having campaigned for Remain ahead of the Referendum, the Hackney MP chose to vote in line with Labour Party three-line whip – even though her own constituency voted 78.5 per cent to remain.
In a statement following the vote, Ms Abbott insisted she still believes Britain is better off in the EU.
She said: “I campaigned for remain and still believe that it is not in our best interests to leave the European Union.
Ms Abbott voted in favour of the Tory Brexit Bill in its final Commons reading
Ms Abbott's constituency voted 78.5 per cent to remain
“However to say that because the vote went the wrong way we are going to ignore the votes of millions of people up and down the country, would be a blow to democracy.
“This does not mean that we have to accept Brexit in the haphazard way in which it is being handed to us.”
Get Quotes on Home Insurance
However the Brexit Secretary has now been forced to apologise to Ms Abbott over a text exchange in which he appeared to say he would not attempt to hug the prominent Labour figure because he is "not blind".
Diane Abbott's most controversial quotes
Mon, February 6, 2017
Outspoken Shadow Home Secretary, Diane Abbott holds strong opinions which does not always go down well.
1 of 11
Being an MP is a good job, the sort of job all working-class parents want for their children — clean, indoors and no heavy lifting. What could be nicer?
According to the texts seen by Mail on Sunday, a colleague of Mr Davis wrote: "Cannot believe you made an attempt to give DA (Diane Abbott) a hug!"
The Cabinet minister replied: "Didn't but the myth grows. I whispered in her ear 'Thanks for your vote' hence the 'F off'. I am not blind."
His colleague wrote back: "Ha! Ha! Thank God you aren't blind. Great week for you and Brexit."
David Davis has now been forced to apologise to Ms Abbott
Mr Davis then made an apparent reference to the Specsavers adverts in which people without glasses make embarrassing mistakes.
A spokesman for the Brexit Secretary has insisted that Mr Davis was joking, and added: "This was a self-evidently jocular and private exchange with a friend.
"The Secretary of State is very sorry for any offence caused to Miss Abbott, someone he has known and respected for many years."
Following the reports, Labour MP Jess Phillips said: "You'd have hoped this sort of misogynistic, sexist attitude had gone out in the 1950s."