Is it acceptable for someone to not like being told "I love you" and prefer other ways of expressing affection?

What is a polite way to communicate to someone that you no longer want to drink with them due to feeling uncomfortable and tired of their drinking habits?
My ex and I broke up a couple weeks ago and stayed friends, but I've been having a hard time keeping this up. I miss him a lot, and I don't know if I should tell him or not?