Can chasing big goals and avoiding being average actually lead to a happier life, or does it come with downsides?

For people who say they have trouble seeing a physically disabled person bullying another person, they can and will and be let off because they are disabled. Why do you think this?