What sort of things harm a persons confidence/mettle?

Somehow I think it's not always as obvious as being put down.
It's like an "energy" that people can steal sometimes.