How can you tell if a guy is being just touchy feely or if he's attracted to you?

If he goes straight for boobs or ass, he's touchy feely. I would say that he is attracted and worthy of you if he keeps his hands to himself and gradually works his way up to meager affection.

A man that is genuinely interested in a woman will treat her deservingly and surely won't risk putting you off in such a way.