Why do people say it's vain to love yourself?

Buttt when you go, "Oh I feel ugly today" they go "NOOO you're beautiful! Don't be hard on yourself!" I mean, there is a point where a person is too vain (pics of themselves all over their house or something) but if someone goes "Hey, I actually am sexy or have a pretty face" they get attacked! It's happened to a lot of people I know, including me... and I grew up ugly so I'm trying to be more confident.