What's the right way to tell someone (a college friend) that you don't want to hang out with them?

We have nothing in common and everything he says gets on my nerves i don't see a positive reason to hang out with him.