Cancer and Virgo

How to be happy in a relationship??
i have a fight with my partner, all started from him meeting his ex wife to talked about annulment arrangement in order to speed up the process to be with him as a free man again..
but i got so mad that he only inform me after meeting his ex wife, ( reason is i dont want to find out by myself).
He told me meeting up with ex wife is to press her to do annulment but she refused to (do it for me and him as i always said he is not a free man).
Now he text me telling me sorry he've got nothing to hide from me, dont know why i am so mad at him.
after texting explaining to him the reason, but he seems to get depress after and wanted to let go..
yet i know he dont want to let go at the same time.
He say in the text, he done nothing wrong why am i keeo pushing him away..
now he is being distance and i already text him and said i over react but he text lesser than usual.

what yall think should i go on or move on???