I think I’ve finally got it.
Height will be an advantage when girl X does not know you.
It’s like a boost at first impression. Just like short height is a curse at first impression.
However, if you are smart, strong, nice to be around, not a simp (polite but minding own business) and not insecure, then the girl will over time naturally gravitate towards you.
This is why it makes no sense to be mad over, as it’s just a first impression boost.