Well it’s not really a big issue in the scheme of things, is it?

And if he thinks you’re the sort of person to blow things out if proportion he might leave it be.

He just likes his happy birthdays spread out a little

It seems to me, that resolving that the FIRST time it happened, might have been easier than... 3 years later. Lol! I wouldn't see the harm in saying, "Thank you, but my actual birthday is tomorrow." I don't think that would have caused any hard feelings...

maybe he's impressed that you remember well enough to say it, but don't know him well enough to know when it actually is.

In my paltry estimation the question is one of value. They hold you in high regard. That to me is actually.....your own personal gift and reward.

Some people get embarrassed over the smallest things.

Omg it's the end of the world...

I can't see a problem.. Its nice you remembered, and all he needed to say is, "thank you so much, but actually it's tomorrow not today"

Your coworker is silly. Don't waste time worrying about silly people. Thats how you have to live, if someone is full of nonsense thinking just tell them once, " hey you should have told me all this time".
Then move on. Its a non issue and no big deal.