Weird neighbor

Few days ago my husband traveled to his home country to see his family and friends, he often travels once every few months to see them, we met online and in the beginning he was visiting then he started living with us but still visits his family and friends from time to time, we were lucky he doesn't do 9-5 jobs so he can stay with us and visit them at the same time, anyways we have neighbors we share a wall with, and usually when my husband is here I never see or hear anything from them, it's most a guy by himself and another guy who I think lives with a woman who come and goes, anyways they never talk to me when my husband is here but when he is away they try to start conversation, yesterday my son was acting up and refused to go inside wanted to keep playing outside and I was very tired so this neighbor came out and started talking to us asking what happened and all and I didn't answer I was too tired, he stayed outside the door watching us untill we went in even after which was weird because when my husband is home and my son does this he never approaches us, and same with the other neighbor he only speaks to us when they know he is away traveling, I was on call with my husband and I asked him about this and he said the other guy runs off when he see him and one time he even tried to say hello and that guy just ran inside as if he was scared, we are also not very social we rarely say talk to anyone here just maybe hi and bye but they always talk to me when he is away sometimes they even ask me about when he is coming back as if they were watching him and knew he left , we just thought it was odd that is all, my husband said they might be scared because he is tall and big and looks aggressive but he is actually quite nice and likes helping people if they ask him.