After he goes or tomorrow, let your neighbours know about the guy and that he's harassing you. And if said guy comes by again to help out with getting him to leave.
Odds are they will do something if the need arises. If they help, thank them with food.
