Mourners joined a host of religious leaders in a minute's silence just after 6pm on Monday in Potter's Park opposite the Tower of London, just down from where terrorists killed seven people and injured 48 on London Bridge and in Borough Market on Saturday night.
London Mayor Sadiq Khan condemned the attack and said the terrorists "will not win".
He said: "As a proud and patriotic British Muslim your perverse ideology has nothing to do with the true values of Islam and you will never divide our city."
He thanked the emergency services and the "brave Londoners" who helped save so many people from the brutal attack.
He was flanked by home secretary Amber Rudd and shadow home secretary Diane Abbot, with leaders of different religions, including the Archbishop of Canterbury Justin Welby, and police chiefs behind them.
The Mayor, added: "London will never be broken by terrorism."
Sadiq Khan led a vigil for the London Bridge attack victims on Monday night
The vigil is the first time Diane Abbott has been seen since Saturday's attack, with voters expressing their anger this morning Ms Abbott was kept out of the spotlight after shadow foreign secretary Emily Thornberry was put forward for several interviews instead of Labour's home secretary.
However, she did not speak at the vigil.
Mr Khan added he was "very angry" about the attack.
The vigil was the first time Shadow Home Secretary Diane Abbot has been seen since the attack
The vigil started as Scotland Yard named two of the three terrorists who were shot dead by police at 10.16pm on Saturday after they mowed down at least 20 people on London Bridge with a van then went on a stabbing rampage in Borough Market.
Get Quotes on Home Insurance
Khuram Shazad Butt, 27, a British citizen born in Pakistan, and Rachid Redouane, 30, who had claimed to be Moroccan and Libyan, both of Barking, are believed to be the two attackers.
Police are awaiting "international" permission to confirm the name of the third man
Many of their victims remain in a critical condition in hospital.
Mr Khan said he had "better and more important things to focus on" when questioned about US President Donald Trump's tweet criticising him just hours after the attack.
Mr Trump took the Mayor of London's tweet saying there was "no reason to be alarmed" by an increased and armed police presence in London out of context, accusing him of saying people should not be alarmed by the attack.