The "literally everyone can use any level of magic" argument breaks every conceivable fantasy world because then nobody would have to toil in the fields, herd animals, bake bread, ask anyone to fetch anything and it would be impossible to have any kind of system or structure in place that can account for literally every tom dick and cunny being a magical bastard