I'm not a brit, but my country has joined EU and as far as the good that comes out of it goes - like free visum to EU countries - it's not likable that they want to control what we do in our area, even when it's proven there's nothing suspicious going on.
About the Brexit i think we should worry if you guys leave the union - France and Germany aren't reliable countries if you know what i mean. Even if they pretend to, or if we believe so.