Tell him but pretend that you're sleep-talking when you do it, so as to maintain deniability by pretending you were talking to some kind of dream lover.
@Montanaman but he never tries to hang out with me other than sex, so I really don't know if he does. Plus he doesn't talk to me almost daily like someone would when they like you