Such is the anger with the Speaker at senior levels of government, it has been suggested he could be blocked from getting a peerage when he retires.
Ministers are furious at what they see as John Bercow’s “bias” during Commons debates on Brexit.
The move would break a tradition dating back 230 years, that former Commons speakers are automatically offered a seat in the House of Lords.
A Cabinet source said: “It’s a good job peerage nominations are in our gift.
“I’m sure we’ll be thinking carefully about which individuals we would choose to elevate to the House of Lords.”
The source told me: “I can’t imagine we would look favourably on those who’ve cheated centuries of procedure.”
By tradition, retiring Speakers have stood down as MPs at the same time, triggering a by-election.
They are then awarded a peerage at the request of the Commons, in a motion asking the Queen to “confer some signal mark of Her Royal favour upon” them.
After a recommendation from Downing Street to Buckingham Palace, they then sit in the Upper House as a crossbencher, an independent.
This is what happened the last time a Speaker stood down.
Michael Martin was ousted ten years ago over his handling of the expenses scandal.
But, despite criticism at the time, he became Lord Martin of Springburn.
Lord Martin died last year.
John Bercow has faced criticism from MPs for months.
In October he told friends he intends to stand down as Speaker this summer, following a report which condemned a culture in Parliament in which abusive behaviour was “tolerated and covered up.”
Mr Bercow has also faced allegations of bullying, which he has strenuously denied.
Earlier this month, plenty of people within the government were furious with the Buckingham MP, saying he broke precedent and ignored the advice of his officials in a heated row about parliamentary procedure over Brexit.
Critics within government saw this as proof that he was unable to be impartial over Brexit and was determined to make life difficult for the government.
Two years ago, Mr Bercow revealed that he voted Remain in the EU referendum.
Asked about the prospect of being denied a peerage when he retires, a spokeswoman for the Speaker declined to comment.