Should I be concerned?

I live on a small farm on the edge of a very small town. In a neighbors field behind us a man walks out of the brush at dusk. He walks out into the open field and then disappears back into the brush as soon as he sees we've noticed him. This happens about 2x a week or maybe more. The owner of the property is an unfriendly elderly man so I'm told that lives on the far side past the brush. The man that comes into the field in the evening is probably in his late 30's or early 40's. He doesn't appear to be hunting and its not hunting season. If he is supposed to be there why does he keep hiding? Also past the brush down a ways from the property owners house is a power station. Someone could easily park there and walk into the field. The other evening when we drove by it there was an older beat up pick up truck parked behind the power station. I dont know why I dont have a pic of him out there.[image deleted]