I Am Naturally Thin and People Think I Am Anorexic

I'm sick and tired of being criticized for being slender. So what if I'm naturally slender? At times I wish I was thicker. I'm 5'6, small built, slender at 134lbs. I'm also fit. I get told I'm too skinny and I need to eat more. I stay active and some bigger girl told me, "Why do you need to work out? You're a stick and you're ridiculous. Jeez!" Just be because I'm slim doesn't mean I don't need to not be active. Does it really matter? Do you get that sometimes?