Obviously we don't know any of the details of your situation but it does sound like the relationship isn't working as either of you expect. You should never feel scared of someone's anger, or feel you can't express yourself. It's liable that he's misinterpreting you if he's getting angry and suggesting you are trying to stop him doing what he wants. Unfortunately, when communication breaks down this way, it's hard to get back on track.
I don't know if it'd work but maybe writing each other a letter when you're both in a very calm frame of mind, saying what you like about each other, but also what is troubling you both, would at least give you both an insight into what is really going on here.
