With a vote of no confidence looming after a controversy-strewn tenure in the role, Tory MPs questioned how Mr Bercow could now be seen as neutral during parliamentary Brexit debates, while the Opposition rallied to his defence.
Leader of the Commons David Lidington warned there would be "strong" reaction to Mr Bercow's latest outspoken move, as he insisted the Speaker must command the confidence of the whole House.
A possible compromise solution was signalled by former culture secretary John Whittingdale, who suggested Mr Bercow's natural term of office was coming to a natural end.
Speaker John Bercow is facing further calls to resign from Tory MPs
Mr Bercow was plunged into fresh controversy after a video emerged of him talking to students at Reading University on February 3, in which he said: "Personally, I voted to Remain.
I don't think John Bercow's position as Speaker is tenable any more
Alec Shelbrooke – Tory MP
"I thought it was better to stay in the European Union than not."
In the video, obtained by The Sunday Telegraph, Mr Bercow says immigration has been a good thing for Britain.
He also referred to "untruths" during the Brexit campaign, how "promises were made that could not be kept", and expressed hopes that Parliament would maintain changes to working hours and health and safety protections after Brexit.
REMOANERS: These people hate BREXIT!
Mon, January 16, 2017
Remainers are finding it hard to accept Brexit.
Get Quotes on Home Insurance
1 of 14
Mr Bercow drew fire from some for branding President Donald Trump a 'racist and sexist'
After Mr Bercow drew fire from some for branding President Donald Trump a "racist and sexist" as he effectively banned him from addressing Parliament during his state visit, a number of Tory MPs said his position as Speaker was no longer tenable when the Commons returned from recess on February 20.
Mr Lidington told the BBC's Andrew Marr Show: "There will be strong reaction among some MPs to what he said at Reading, particularly after what he said about the state visit earlier in the week.
"Ultimately, the Speaker has to command the confidence of the House of Commons as a whole.
"John has his very strong supporters as well as his strong critics in the House of Commons, but we shall have to see how members as a whole respond."
The parliamentary website states: "The Speaker is the chief officer and highest authority of the House of Commons and must remain politically impartial at all times.
"On election the new Speaker must resign from their political party and remain separate from political issues even in retirement."
Mr Bercow's spokeswoman said how the Speaker cast his ballot in the Brexit poll, or Strictly Come Dancing, had no impact on his ability to deal fairly with all MPs.
John Bercow told students at Reading University, ‘Personally, I voted to Remain’.
She said: "Mr Bercow voted in the EU referendum, along with millions of others.
"The record shows that he has rigorously facilitated the raising of concerns of those on both sides of this argument, as he does on every other issue.
"The Speaker's impartiality is required on matters of debate before the House, and he has been scrupulous in ensuring that both sides of the argument are always heard."
The spokeswoman said Mr Bercow's record showed he was neutral in the chamber "irrespective of how he voted in a referendum, general – one would hope for himself – local, or Strictly Come Dancing."
James Duddridge, who tabled the motion of no confidence in the Speaker over his Trump comments, insisted that the Speaker is is "no longer impartial".
Speaking on BBC Radio 5live's Pienaar's Politics, he said: "I think there will be a vote of no confidence and I think he will go.
"There's absolutely no way Speaker Bercow can sit in the chair on European issues.
Being President Trump: Key Moments
Fri, February 3, 2017
Inside Donald Trump's first week in the White House.
1 of 14
Trump signs an order to review the Dodd-Frank Wall Street to roll back financial regulations of the Obama era.
"When you become Speaker you must be impartial.
"He's no longer impartial, he's no longer able to continue to do the role, which is why I think the House will vote him down in a vote of no confidence.
"In reality he may see the lie of the land and go before he's pushed."
He added he had been "amazed" by the number of people to privately voice to him their support for his motion.
Mr Whittingdale told ITV: "John was elected with a very firm pledge that he wouldn't stay for more than eight or nine years, and we are pretty much getting close to the end of that period.
"So I wouldn't expect him to stay for much longer."
Tory MP Alec Shelbrooke told the BBC: "I don't think John Bercow's position as Speaker is tenable any more.
"He has to resign."