Don't forget that he got mast multiple layers of security.
You have to swipe a badge at the elevators to allow you to access specific floors as well as the doors on those floors.