What if somebody switches and gets all serious on you like..."hey that's not a joke"?

Id Apologise and just joke with them if its to their taste. Different people have different senses of humour usually before i make a joke that could be offensive i wait until i know the person and if it would be offensive to them, for instance, if theyve made joke in the past to me that was a bit dark. Some people also just cant handle a joke

I guess some things aren't funny to some people.

That's something I would do.

Then I apologize but maybe they need to chill

I usually Apologize but then if they keep harping on about it ...it's just unnecessary and annoying

It happens all the time with me which is what I stoped talking to certain people because they are stupid.