Members said they wanted to attend the House of Commons to show a commitment to freedom and democracy, and get "back to business".
Thursdays are often a quiet day in Parliament, with MPs travelling back to their constituencies or making trips elsewhere.
But the chamber was packed for a minute's silence at 9.33am to commemorate the victims, followed by questions, before Prime Minister Theresa May makes a statement on the attack later.
MPs remain at Westminster to show 'commitment to democracy' following yesterday's terror attack
Senior Tory MP Neil Carmichael said he was due to travel to Peterborough but had stayed in Westminster because "you've got to be in Parliament on a day like this to demonstrate that total commitment to getting back to normal as quickly as possible".
Mr Carmichael described the mood among his colleagues as "resolute" saying: "We're going to have a statement from the Prime Minister at 10.30am and everybody, I think, is absolutely determined to demonstrate commitment to liberty and democracy whilst remembering what happened yesterday.
"A policeman was killed, two other innocent people were killed, so that's not to be forgotten."
London Terror Attack: LATEST PICTURES Thu, March 23, 2017
5 dead and 40 injured in Westminster terror attack
Twitter 1 of 42
Concrete blocks en route to reinforce security in London
It's very moving coming back to a place that was under attack less than 24 hours ago
Senior Tory MP Neil Carmichael
A sombre calm hung over Westminster the morning after the attack and there were none of the large groups of tourists usually seen taking photos outside Big Ben.
Get Quotes on Home Insurance
A large area, including Parliament Square and Westminster Bridge, remained cordoned off and a Union flag flew at half mast above the Palace of Westminster.
Westminster underground station was closed except for interchange between lines, but Parliament remained open.
The police presence was heavy with dozens of officers, many armed with guns, keeping guard at the entrance, as MPs, peers and staff flashed their passes and filed quietly into an entrance at the side of the building.
"It's very moving coming back to a place that was under attack less than 24 hours ago," Mr Carmichael said.
"The one thing about working in this building as long as I have now, you know so many people – not just the MPs, but people who are in fact looking after us in terms of security, policemen, the catering staff, library staff and all of that.
How the front pages of newspapers are reporting the London terror attack Thu, March 23, 2017
Media across Europe, to the USA, the Middle East have paid tribute to the victims
1 of 20
"So we know each other and we watch out for each other, never more than just in the last 24 hours."
Despite the shock of the attack, MPs were determined to get back to work.
Barry Gardiner, shadow international trade secretary, told the Press Association: "Everything has to be seen to get back to normal as quickly as possible so that people realise that this sort of attack is ultimately a futile gesture.
London Terror Attack: Politicians react Thu, March 23, 2017
Politicians from around the world have responded to the London terror attack which led to the death of 4 people and more than 40 injured
GETTY/REUTERS/TWITTER 1 of 12
At least four people have been killed including a policeman
"I think the police have been absolutely right not to release details of the attacker, to give it no publicity.
"What happened yesterday was sordid, shameful and vile.
"The attacker caused the most appalling carnage in the lives of 20 to 25 families.
"I think everyone in Parliament will want to pay tribute to the security services and the police and how they responded yesterday, and the officer who tragically died."
The chamber was packed for a minute's silence at 9.33am to commemorate the victims
SNP MP Alan Brown told the Press Association on his way in: "I feel a slight sense of apprehension, but we can't let the crimes of one person prevent the working of Parliament.
"Obviously, our thoughts are with the family of the policeman and other people who were killed.
"I suspect it will be an emotional day, it will certainly be a very sombre day, in Parliament.
"Today it is back to business. I've got a question on the order paper and a constituency matter to raise, so it is back to the day job of being an MP."
A large area, including Parliament Square and Westminster Bridge, remains cordoned off
Mr Brown had just voted in the division lobbies when the attack took place and he sought refuge with a group of other MPs in the members' tea room, where they sat glued to the television as events unfolded.
He praised the bravery of Tobias Ellwood, the Foreign Office minister who ran towards the gunfire and tried to save the life of Pc Keith Palmer, who died from his injuries.
Mr Brown said: "I think that was so brave of him. I saw the pictures showing him with blood on his face having tried to give CPR. It is also a testament to the training he has received."