Sir Keir Starmer has insisted he took “decisive action” over comments made by Rochdale by-election candidate Azhar Ali about Israel and Jewish people.
Labour has been criticised for initially standing by Mr Ali after it emerged he claimed Israel had “allowed” the 7 October attacks by Hamas.
On Monday the party said it was withdrawing support for him after further remarks came to light.
The Labour leader said this was a “tough” but “necessary” decision.
Speaking on a visit to Wellingborough, Sir Keir said: “Further information came to light yesterday calling for decisive action, so I took decisive action.
“It is a huge thing to withdraw support for a Labour candidate during the course of a by-election.”
He added: “But when I say the Labour Party has changed under my leadership I mean it.”
Mr Ali will still be listed as the Labour candidate on the ballot paper at the by-election on 29 February, as under electoral law it is too late to remove him.
However, it is understood he has been suspended from the party pending an investigation.
Over the weekend, Mr Ali apologised for comments he made at a meeting of local party members and councillors, which were reported by the Mail on Sunday, claiming Israel had “allowed” the Hamas attacks as a pretext to invade Gaza.
Labour initially defended him as a candidate, saying he had apologised and retracted his remarks.
It was not until Monday evening that the party announced it was withdrawing its support for Mr Ali as its candidate, saying further comments had emerged.
The Daily Mail published a fuller recording from the meeting, allegedly of Mr Ali, blaming “people in the media from certain Jewish quarters” for the suspension of Andy McDonald from Labour.
Martin Forde, who led a review into allegations of bullying and racism in the Labour Party, said the handling of the case had been “pretty shambolic” and it would have been “sensible” to withdraw support for Mr Ali when his comments first emerged.
The senior lawyer told BBC Radio 4’s Today programme there was a perception among some left-leaning MPs “that when it comes to disciplinary action taken against them then things move rather slowly, but if you’re in the right faction of the party, as it were, then things are dealt with either more leniently or more swiftly”.
Responding to Mr Forde’s comments, Sir Keir said when he became leader he had promised to “tear antisemitism out of the Labour Party” and had “ruthlessly” changed the party since then.
“It’s made no difference to me where somebody stands in the Labour Party,” he added.
Sir Keir was defended by former Labour MP Lord Mann, who is now an independent adviser to the government on antisemitism.
“I think it’s actually quite a bold decision, basically to throw away a parliamentary seat,” he told the BBC.
“I think the Jewish community, as it reflects, will take quite a lot of comfort from the fact that Sir Keir Starmer has been prepared to do that.”
However, the Conservatives have accused Sir Keir of only acting due to media pressure.
Housing Minister Lee Rowley earlier told the BBC that the situation was a “complete mess”.
“Azhar Ali did not just make some of these statements quietly to a few people in a corner,” he said.
“Some of these statements that have come out, that have had to be dragged out by the press, were made to Labour Party meetings.”
Asked whether others present at the meeting where Mr Ali’s alleged comments were recorded would also be investigated, Sir Keir said: “Any allegation that needs to be investigated will be investigated.”
Labour’s decision to withdraw support from Mr Ali, who was selected to represent the seat after the death of Sir Tony Lloyd, means it is also highly unlikely that he would be chosen by the party to fight this year’s general election.
And it will add an extra layer of uncertainty to the outcome of the by-election on 29 February in Rochdale, where Labour has a majority of more than 9,000.
Also running are former Labour MPs Simon Danczuk, now the Reform Party candidate, and George Galloway, of the Workers Party of Britain, who has campaigned against Labour’s stance on Gaza.