I don't see what is wrong with having a preference with physical attraction. While the person as a whole is of course important, physical attraction also plays a role in a relationship. If it isn't there then it isn't there, and you shouldn't have to apologize for having a certain type.
Either way though you lose. If you don't respond you become the bad guy, if you tell him right off the bat that you don't find him attractive, you are also the bad guy. I think it is best to not hurt his feelings, and just ignore him. In his eyes you will just be a bitch, at least you aren't a mean bitch lol. There will be someone else out there who will find him attractive, just not you.
