I need actual serious advice.

I am making a 3d mmorpg in c++, Should I care about optimization before the bottlenecks start to appear, or should I only care about it when they show up?

obvious optimizations like broad and specific collision and heightmap interpolation are already at play.