You're assuming it's not "turtles all the way up".
You're ALSO ASSUMING that everyone approaches spacetime like being forced to roll down an infinite ramp.

Are 5th directional beings subject to 4th directional choices? Are you directional beings ensnared in only the lowest energy paths 6th directional beings create?

Let's simplify.
Imagine all possible realities like channels on a TV set. A 4D being might stroll through a bit of Channel 3 then a bit of channel 4 then a bit of channel 5 all to go from the left front side of the TV screen to the back right side of the screen.

Using the same scenario.
A 6D being might follow the same path, but go from 7 days from now to the day before, and so on using the same physical path until they are 14 days in your past.

You assume time is linear.
Therein, what happens if creatures don't care about that and see the entirety of your existence as a thin membrane to walk through effortlessly.

Another way to think about it is to imagine a village square in full 3D seen from overhead. That represents a single instant moment of your existence. Beings not bound by your conception or time may simple stroll through everything in the scene unnoticed and untouched in that single instant.

You're vaguely grasping the implications of temporal manipulation and observation, but you're ignoring the fact that these being have far more directions of choice than you.

Let's hit another poor analogy. Your life is a comicbook showing 2D slices of your existence as frames. However, this comicbook is mass produced. Higher directional beings can enter your SINGULAR comicbook instance, create all sorts of chaos, but the mass produced copies? Completely untouched by that singular higher directional being as they exist as individual copies of a singular thing. Is it sinking in yet?