The memorial is being led by Mayor of London Sadiq Khan, who yesterday vowed the city would “not be cowed by terrorism”.
Huge crowds are now filling the famous square to listen to speeches and remember the victims of the attack.
Mr Khan said the response of the public and emergency services showed why London was the “greatest city in the world”.
He said: “When Londoners face adversity we come together. We stand up to our values and we show the world we’re the greatest city in the world.
"Our response to this attack on our city, on our way of life, on our shared values, shows the world what it means to be a Londoner.”
Mr Khan said: "We come together as Londoners tonight to remember those who have lost their lives and all those affected by the horrific attack yesterday.
“Also, to send a clear message, Londoners will never be cowed by terrorism.
“Those evil and twisted individuals who try to destroy our shared way of life will never succeed, and we can condemn them.
A huge crowd is paying respect to the victims of yesterday's London attack
“Our hearts are with the family and friends of PC Keith Palmer of Aysha Frade, Kurt Cochran and all those injured in the attack yesterday.
“The victims were people who came from all corners of our world, and this is a time to express our gratitude to the heroism of our police officers and emergency services who ran towards danger to help, and at the same time they encouraged others to run to safety.
People light candles during a memorial to those killed or hurt in yesterday's terror attack
Get Quotes on Home Insurance
A memorial vigil is being held for the victims of yesterday's attack
“London is a great city full of amazing people form all backgrounds and when Londoners face adversity we always pull together.
He was joined by Acting Met Police Commissioner Craig Mackey, who said: "This cannot be undone, much as we would wish it.
"We do get to choose our reaction and gathering here tonight shows exactly how we must move forward. We must stand together.
London terror attack victims Thu, March 23, 2017
ISIS claims responsibility for London attack that killed at least three victims
1 of 10
"People have tried to tear this city apart in acts of terror many times before. They have never succeeded and they never will.
"We will act as one and embrace our differences.
"We will clearly demonstrate the futility of hatred and the power of friendship."
READ MORE: When is the London terror attack vigil?
The crowd at this evening's memorial vigil
Hundreds of people are gathering at Trafalgar Square to pay their respects to the attack victims
Amber Rudd, Home Secretary, paid tribute to PC Keith Palmer, who lost his life preventing the terrorist from entering Parliament.
She continued: “They will not win. We are all connected and today we showed that by getting about our normal business.
"Because the terrorists will not defeat us. We will defeat them.
"We are strong in our values and proud in our country.”
Members of the public paying their respects to the victims this evening
Three people were killed by terrorist Khalid Masood yesterday: PC Keith Palmer, who died stopping the attacker entering Parliament, US tourist Kurt Cochran, and mother-of-three Aysha Frade, who was killed as she walked to pick up her children from school.
Forty people were also injured in the car and knife attack, several seriously, including three other police officers.
Masood was shot dead by an armed Met Police officer at the scene just metres from the House of Commons.
Cross-denominational religious leaders arriving at the vigil this evening
The violent criminal had driven a 4×4 car across Westminster Bridge, striking a crowd including schoolchildren, before ramming the vehicle into a gate outside Parliament.
He then attempted to storm the building, fatally stabbing a police officer before being killed himself.
This is a developing story. Check back soon for the latest photos and information.