Back in the middle ages they were. That's what the word "lubber" means. A lubber is a castrated fool or retard as they called them. It kept them out of trouble. When sailors would return from voyages they'd see all these fat castrated retards wandering peacefully and aimlessly about. Thus "land lubbers"; sailors were the most athletic and courageous men of their times and they regarded all land dwelling men as "Land lubbers".

(Middle Dutch):
Meester snyt die keye ras
Myne name Is lubbert Das

(English):
Master, cut the stone out, fast.
My name is Lubbert Das.