If there is a nice fence, (and I hope there is), you should be safe...but the barking...terrible. Obviously the dog believes you have invaded HIS territory.
It is your neighbor who must address the territorial behavior in his own dog. Only he, as the dog owner, can cause the dog to be trained not to be aggressive.
@CastorTroy Dogs sense fear. They can smell it. If your neighbor is afraid of his own dog, he has no business owning it, because he won't be able to control it if there is a problem.
If there are ANY problems you need to start calling it in to animal control, or the police.