They care but mostly in countries where the average is relatively tall. As a 190cm I'm not exceptional but definitely tall enough to attract women. In a country where average man is 170cm or below I would probably not get as much female attention because I would not fall within the golden standard and would be perceived as too tall.