Why is interracial relationship/marriage/sexual affair etc. a taboo?

I'm white, that's my reality I think it really comes down to location. Some places no one looks twice. It's a non-issue. Other places they look four times. And it's not just coming from white people. It's weird to get attention over something that you don't even consider is attention worthy. Easy to write off though. I've never had anyone say anything but it would be a pretty short conversation if they did.