So even if we don't open the door to time, how do we actually beat Ganondorf? That dude took on a fully-armored Hyrule Castle and won BEFORE he got his Triforce powers. Kid Link tried fighting him, and get blown the fuck out by Ganondorf's hand laser.
I always figured Ganon in Ocarina functioned like Sauron Lord of the Rings. Where Sauron was going to take over middle earth anyway, but getting the ring would have guaranteed victory. Adult Link just cleans up the mess of a conquered Hyrule.