Do you think there are a lot of people who avoid "nerdy things" because they're "nerdy"?

I wonder how many people are seriously missing out on not learning anything about relativity and quantum physics just because it sounds tricky and nerdy.