A stray cat has decided to make my garden his home. He's a kitten and his mom seems to have abandoned him.

I don't know what to do.I am not a cat person. I actually have a strong dislike for them. Never really liked animals too much anyway.
We give him milk twice a day. But... He climbs the window grill peers inside and keeps meowing a lot. And if we happen to get outside the house and he spots us he keeps meowing non stop.
I don't understand him. What does he want?