Recently released figures showed that the prison population in England and Wales was at its highest-ever level.
But with concern about released prisoners reoffending, the government insisted that no violent offenders or domestic abusers would be eligible for early release.
Business minister Jonathan Reynolds told reporters that it was a difficult decision to release the 1,700 prisoners.
Downing Street has said the policy was required to avoid unchecked criminality, where recently convicted criminals have been spared jail sentences because there were no places available.
Reynolds placed the blame on the last Conservative government, which was voted out in July, saying the decision should have been made prior to the election” but that it had given up on governing.
Chief Inspector of Probation Martin Jones told newsmen that the pressure on prisons was extreme.