How do you get rid of the toxic belief, that if you think well of yourself.. you are arrogant?

By working on the realisation that thinking well of yourself is part of Self Love. It is only pointed out as arrogance by people who feel discomforted by other peoples light. I am not talking about the kind of belief which thinks one is superior than another. In fact, thinking well of oneself doesn't take away from how well others can be too.
Get into the practice of praising yourself for every little achievement, every thing you like about yourself. Practice until it becomes muscle memory to think this way.