Mohammed Shafiq, chief executive of the Ramadhan Foundation, said those voicing support for extremism were out in public “hurling their abuse”.
Mr Shafiq added the police and terrorist prevention agencies should step in to put an end to the “glorification” of “barbaric acts” and not leave it to Muslim communities.
Speaking on the Today programme on BBC Radio 4, he said terrorist sympathisers should not be left to “roam our streets”.
He said: “You can’t point the finger at the Muslim community as the Prime Minister did and say we Muslims are too tolerant of Islamic extremism when actually it’s the society and its the agencies that are there to protect us from these terrorists are allowing these terrorist to roam our streets.
Mohammed Shafiq has called for the immediate arrest of those voicing support for terrorist acts
If you glorify the barbaric crimes which Isis have been involved in then the law should take its course
“They’re out there. They’ve been outside mosques, they’ve been out there outside Edgware Road and other parts of the country hurling their abuse.”
Asked if Muslims who hurl that abuse should be “locked up”, he added: “If you glorify terrorism, if you glorify the barbaric crimes which Isis have been involved in then the law should take its course and you should be held accountable.”
The number of potential UK-based terrorists on the radar of the British security services is as high as 23,000, it has emerged.
The figure is more than six times the figure previously released by the Home Office at the time of last month's Manchester bombing.
Morning after London attack: Heightened police presence at Borough Market & Bridge Sun, June 4, 2017
SIX people have been killed after a horrific terror attack targeted London Bridge and Borough Market on Saturday night. These are the pictures are London wakes up to the tragic scenes
Play slideshow 1 of 23
Get Quotes on Home Insurance
Then, it was revealed the security services had 500 live investigations into more than 3,000 suspected radical jihadis, including about 400 people who have remarkably been allowed to return to our shores after fighting with terror group Islamic State in Syria.
But, it has since emerged a further 20,000 radical Islamists have been considered a "person of interest" to the security services at anyone time, according to a security services source.
It is not clear over which timescale the figure relates to.
But the news has led to fears there are simply not enough resources for our security forces and counter terror police to monitor all their activities at anyone time.