Public servants receive a 10% "premium" over the private sector, Philip Hammond said – as he warned ministers against leaking cabinet talks on the pay cap.
The chancellor refused to comment on reports he had said at a meeting that public servants were "overpaid".
And he suggested some colleagues who do not agree with his approach to Brexit were trying to undermine him.
International Trade Secretary Liam Fox said he "deplored" the briefing by some of his ministerial colleagues.
Mr Fox told the BBC's Sunday Politics they should "be very quiet" and "stick to their own departmental duties", adding: "Our backbenchers are furious and the only people smiling at this will be in Berlin and Paris."
Since the general election, cabinet splits have surfaced over the issue of the 1% cap on public sector pay rises, with some ministers pressing for it to be lifted.
Labour is promising £4bn which it says would offer a pay rise to workers.
On the BBC's Andrew Marr Show, Mr Hammond defended his stance, saying public sector pay had "raced ahead" of the private sector after the economic crash in 2008.
While in terms of salary alone, that gap had now closed, he continued, when "very generous" pension contributions were taken into account, the 10% disparity between public and private salaries was a "simple fact".
Asked about a Sunday Times report claiming he had said the former were "overpaid", the chancellor insisted he was not going to discuss what was and wasn't said in a cabinet meeting.
"I do think on many fronts it would be helpful if my colleagues – all of us – focused on the job at hand," he said.
He added: "If you want my opinion, some of the noise is generated by people who are not happy with the agenda that I have, over the last few weeks, tried to advance, of ensuring that we achieve a Brexit which is focused on protecting our economy, protecting our jobs and making sure that we can have continued rising living standards in the future."
Mr Fox, one of the leading Brexit campaigners in the cabinet, rejected press reports he had clashed with Mr Hammond on the issue, saying the two had a "very good working relationship".
"I don't know where the briefing is coming from, but I do know it's got to stop," he said.
Get Quotes on Home Insurance
He added: "I think there's too much self-indulgence, and I think people need to have less prosecco and have a longer summer holiday,"
Pay rises for most public sector workers are set by independent pay review bodies, but have effectively been capped at 1% each year since 2013.
Before that, there was a two-year freeze on pay for all but the lowest-paid workers.
The government has come under pressure over the policy since the general election, with some Conservative ministers speaking out in favour of lifting the cap.
Shadow chancellor John McDonnell said Labour would spend £4bn on ending the cap, insisting this would be enough to give a real-terms increase for public sector workers.
Pay review bodies would be asked to come up with an "honest judgement" and a Labour government would follow their advice, he said.