What’s The difference in being lactose intolerant or being allergic to milk ?

I can eat ice cream, I can eat yogurt I can eat cheese mostly. I just can’t do milk and makes me sick even while I am eating it. What do you think ?