How are people from Italy considered "Not Caucasian" by USA until the '50's, even tho the Roman Empire was the birthplace of modern western civilization and governance?

Was it because of their Catholic religion or their orange skin?