It's pretty straightforward. The machines can't actually built a perfect Matrix because no matter what they try there's always glitches present that cause a tiny percentage of humans to break the conditioning, leading to them waking up from the Matrix and being ejected from the system into the real world. Those freed humans inevitably group up and form a society (Zion) and find ways to re-access the Matrix, subsequently freeing more humans. The continued glitches eventually culminate in "The One", an awakened human who is powerful enough to manipulate the Matrix itself and rewrite the code.
The machines have had this happen enough to realise that this whole scenario is inevitable; they can't code a flawless Matrix because they don't understand what humans actually want. Instead of trying to fix the problem (which they inherently can't), they allow Zion to exist as a temporary solution; the awakened humans are of negligible threat to the Matrix until The One has awakened, and once that occurs the existence of the "last human city" has already conveniently corralled all the awakened humans into one location for the machines to exterminate.
The Architect explains all this to Neo and lays out that the previous Ones have all agreed to continue the machines' arrangement (reset the Matrix to a new version that will eventually see the same glitches, as well as reseeding Zion and beginning the process all over again) since the humans have no real way of defending themselves against the machines; Neo tells them to fuck off because he wants to save Trinity instead...