how would you handle?

My wife and I live in a condo. Out neighbor's have two children, boys, a 7 and 2y.o. we have fraternal twin boys, 6. My boys love to play with the neighbor's oldest. However one of my sons has ADHD and gets highly agitated when the neighbor's soon decides he doesn't want to play with him or doesn't want to play a particular game. My other bit is much more easy going but is slighter than the neighbor's son. The neighbor's soon had knocked him to the ground a couple of times and lied about it. M growing tired of having to calm my son's down after this kid does something either of them. the boys father is not in the picture but the mothers fiance is. Unfortunately he had no control over the boy.