By restricting immigration, the government directly intervenes in labor supply, artificially making it scarce resulting in inflated labor costs.

Anti-immigration is anti-free market. If you don't support the unconditional free flow of labor and capital you're not an individualist, but a collectivist.