The MEP for Yorkshire and the Humber from 2004 to 2014 accused Ms Miller of getting involved without having the right to do so.
In a face to face confrontation, Mr Bloom told Ms Miller, who was behind the Supreme Court case which forced Theresa May to seek parliamentary approval for triggering Article 50, to get elected.
The pair met as part of a TV blind date on BBC2’s Victoria Derbyshire.
Mr Bloom said: “You’re in a little bit of a dodgy position, if you don’t mind me suggesting it, because you are very heavily involved in this process of Brexit as a personality.
Gina Miller and Godfrey Bloom clashed over Brexit during their blind date
But you’re not elected, you don’t have a mandate
“But you’re not elected, you don’t have a mandate. Go, wherever you live and say ‘vote for me’.”
The businesswoman hit back saying she had the right to “speak up” over Brexit.
She said: “I never knew that politics wasn’t in the business of everyone. I thought we all had a responsibility to speak up and had the right and the voice to speak up.
“Politics is not something that happens to me. Politics is something that affects every single part of my life, my children’s lives, my friends’ lives.
“So I have every right to stand up and say what I believe. I don’t have to be a politician, as an individual citizen I have a civic duty which I'm exercising.”
Get Quotes on Home Insurance
Godfrey Bloom told Gina Miller she needed to get elected to interfere with Brexit
But Mr Bloom added Ms Miller would “lose some of the stick” if she was in elected office.
The former politician earlier put it to Ms Miller that Britain should leave the European Union without negotiating.
He said: “When I left my London club, I wrote a very nice letter and said it was marvellous, good bye and they said ‘oh dear, sorry to lose you, good bye.’
“I don’t understand this, what are we negotiating?”
Not answering the question, Ms Miller replied it was important to focus on “what happens next”.
Snap election 2017: The pictures politicians may not want you to see Sat, May 27, 2017
Protests, fights and daleks, it's all happening as the politicians hit the campaign trail for the snap election
Play slideshow PA 1 of 48
Jeremy Corbyn during a visit to Hackney Marshes Football Pitches, to highlight Labour's manifesto commitment to ensure 5% of the Premier League's television rights income is diverted to the grassroots game, during a General Election campaign
Ms Miller branded Theresa May's recent debate no-show "totally scandalous" as she joined forces with the Liberal Democrats to recreate the Nigel Farage moment on the Brexit flotilla in a desperate attempt to canvas for votes.
Instead Mrs May sent Home Secretary Amber Rudd in her place to take on Jeremy Corbyn, Tim Farron and other leaders.
Mrs Miller's seized the opportunity to attack the Prime Minister, implying it does not bode well for her negotiations in Europe over Brexit.