Well...
I understand how hard it is when you don't feel like you fit patterns, I feel the same way. This may sound really cliché, but remember that you have other characteristics that define you as a human being.

And also, you can think that if you felt handsome and someone was with you for only this reason, what would be the point? You could easily be traded by someone more attractive. Attractive people are everywhere, now people with other good characteristics are very hard to find.

So, I believe that attractiveness plays a big role when finding someone, but when your head tries to play tricks on you, try to look at another side, you are avoiding more shallow people in your life. And you can be still attractive to someone, by showcasing other great characteristics you have.