Why would someone casually drop how pale you are in conversation?

Not like saying you look ill but actually pointing out your skin colour? This is the guy who wont bugger off now. It's a strange comment in my book.