Do people have trouble telling if you're serious or not?

As someone who is serious 95% of the time, I think when I do attempt a joke, people take it seriously and it backfires.

There's also the small problem of me just not being funny. 😩