Sometimes I'm afraid to disagree or agree in my relationships

It's like a trauma I'll hold onto. The idea that if I disagree or fight with my partner even over the little things that I will be tossed in the garbage. It's a work in progress, but it does cause anxiety.
My current partner loves me even if I disagree or I'm not submissive.