Why would some people assume about others that they only tolerate you and actually not like/accept you?

Like they would talk for others about how they'd secretly be glad as soon as you're out of the door, but they never asked those people or let them speak for themselves.