For guys does being called sir make you feel respected or annoyed?

I have heard ‘Don’t call me sir I work for a living’ I think American children are often brought up to use it to be polite.