Lord Sugar claims he could have swung the 2016 EU referendum in favour of Remain if he had taken part in a live TV debate.
The Apprentice star said he had been invited by former prime minister David Cameron to “take the lead for the Remain camp” on the BBC debate.
But he said he rejected the invitation and was replaced by London Mayor Sadiq Khan.
“To this day,” he said, “I kick myself for turning it down.”
Speaking during a debate about calls for another EU referendum, Lord Sugar referred to the Leave campaign’s controversial claim that Brexit would mean an extra £350 million for the NHS.
“I was fuming that somebody didn’t ask Boris Johnson to put his hand on his heart, look down the lens and tell the British public that the £350 million was a truthful statement,” he said.
Lord Sugar claimed that “in my forceful manner” he would have made Mr Johnson admit it was not true, adding: “And who knows, maybe, that could have swung the vote.”
The £350 million claim, which featured on the side of a Vote Leave bus, was widely criticised because it is an approximate sum for the UK’s “gross contribution” to Brussels.
Mr Johnson has since defended it, and earlier this year said it was an “underestimate”.
In his speech, Lord Sugar also suggested that if company law rules applied to politicians, Mr Johnson and fellow Leave campaigner Michael Gove should be prosecuted for what they said and “putting this country into five to 10 years of post-Brexit turmoil”.
Opening his speech, he initially indicated that he was against another EU referendum, saying: “It would be a complete farce if you could have another crack of the whip.”
But he claimed there was a “good argument” for the 2016 referendum to be declared void if “the public were misled” and ended his speech by backing another public vote.
Campaigners from the People’s Vote campaign want a referendum to be held on the outcome of Brexit negotiations, with one option being to remain in the EU.
During the Lords debate, former Tory chancellor Lord Lamont was critical of the phrase “people’s referendum”, which he argued was “an Orwellian use of language designed to conceal meaning”.
He said: “It would undermine the very basis of democracy because it would suggest a decision by a majority is insufficient to make that decision legitimate.”
But former Liberal Democrat leader Lord Campbell said people’s expectations had been “failed by incoherence and incompetence”.
He added: “The people of Britain have a right to be able to pass judgment on any deal forged in such circumstances.”