There was an anon yesterday or so talking about how the temple was causing lag:
Someone else did an in-depth investigation.

The cause of the lag is netcode, because of all the new inventory updates that occur because of the update.
Basically, the temple items are considered inventory items, even though they're not visible.
You also get minor lag spikes when defeating enemies and getting drops.
You're constantly syncing with the server every frame because of stuff like Suibian Temple expeditions as well as the Lemnian Hollow loot drops.