>>215898700
>>215898730
Just imagine this super simple scenario.
you are hovering in mid air, completely fixed in one place, with a jet pack.

1. if you fly into the orange, you will come out from blue with your momentum conserved. self explanatory.
2. If the box moves into you, you will stay in your place, but the box would be keep moving. to a stationary observer moving along with the box, this is the same as you moving.

"but the only moving thing is the orange in the original scenario."

It's literally the same in this scenario. The orange portal contains an entire identical copy of the world. To the guy, orange portal moving towards him is literally same as an exact copy world moving towards him.