If you're getting angry or annoyed by negative people aren't you being negative?

If you react to them. But if you feel the emotion, regulate and manage it, and handle the situation appropriately, I'd say not.