Has any 'toxic' relationship impact your self esteem in a self destructive way?

Totally. And it can take a long time to undo the damage. I was married to a very controlling and jealous woman and I carried the fear and shame she put on me for a long time. I had to work to get out from under it.