Jeremy Corbyn suggested that everyone pays the price for underfunding criminal justice during his live debate with Boris Johnson.
The issue was a heated one following the recent terror attack on London Bridge, during which two people were killed.
Usman Khan, the perpetrator had been released following eight years in prison for terror offences.
Mr Johnson again hit out at the early release of Mr Khan, and pushed for a removal of automatic early releases.
Mr Johnson said: “I think it’s extraordinary that Mr Khan was out on automatic early release.
“We need to see that terrorists serve their full sentences.”
Mr Corbyn quoted victim Jack Merriit’s father, who had argued for a society of rehabilitation.
Nick Robinson, the debate’s chair, quoted him saying: “We do not lock up and throw away the key.”
Mr Johnson responded: “I have huge sympathy and it was a terrible thing but the judge who sentenced him made it clear it was a risk to public safety.
“We need to get rid of early release. We must put public safety first.”
Mr Corbyn argued that the solution was investment in the police and prison services.
He said: “You have to give people security but security doesn’t come on the cheap and you have to invest in it.
“If you don’t run the prison service so it includes rehabilitation and if a system doesn’t assess people properly before they’re released then that’s a danger to all of us.
“What happened on the streets of London was appalling. But if you underfund the criminal justice system then we will all pay the price with our own security.”
Mr Johnson also made the point that Mr Corbyn will put a Home Secretary in power who wanted to disband MI5.
Diane Abbott signed a bill calling for the abolition of MI5 in 1989 but has since changed her views on the subject.
Mr Johnson also reiterated his claim to put 20,000 police officers back on the street.
Since 2010, police numbers in England and Wales have dropped by 20,500.