The BBC pundit told Football Focus that the game ought to be run more like a ruthless business.
His remarks came after MPs this week passed a vote of 'no confidence' in the FA.
Although the vote has no legal weight, it comes as another blow to the biggest organisation in English football.
The FA has been previously accused of being out of date in the way it is run and the people in power.
Danny Mills dismissed the entire board of the Football Association (FA)
Parliament passed a vote of 'no confidence' in the FA earlier this week
The government has repeatedly called for the FA to reflect modern society and to change the way it makes decisions.
Echoing these remarks, ex-Leeds and England player Mills, who is a member of the FA Commission, told the BBC that the FA needed an urgent makeover.
"If I’m being honest, I think they should all hold their hands up and resign. Just step down and walk away from it," he said.
"Let the FA be run as a business. Run it like a football club – like Arsenal or Manchester City with a chairman at the top.
"Everyone gets a vote on everything. It doesn’t work in business, and it certainly doesn’t work like that in football."
Mils then suggested that David Gill, former vice-chairman of the FA, should be appointed to run the organisation.
Get Quotes on Home Insurance
Mills said the FA board should pack up and go home
Parliament criticised the FA as outdated
If I’m being honest, I think they should all hold their hands up and resign
The FA is run by a group of people called the FA Council, which has 122 members.
However, out of those members, only eight are women and four from ethnic minorities.
On top of this, more than 80 are aged over 60.
The FA has also been criticised for the influence and power of the Premier League.
Jon Motson, who also appeared on the BBC football show, added that the Premier League "seems to have more power than it should".
Jon Motson also criticised the state of the FA
The FA is in charge of appointing the England boss
On Tuesday the FA chairman, Greg Clarke, said he intended to present "substantial" changes in governance, and that if the government did not support them, he would step down.
The MPs complained about the financial power of the Premier League, the failure of grassroots facilities, incompetent club owners at odds with supporters and the paucity of black and ethnic minority coaches.
Those in the House of Commons argued the FA had to be given the remaining weeks until the end of March deadline to comply with their demands.
However, if they fail then Sport England could withhold funding to support grassroots projects.