Have you ever had a song or band you love tainted by the memory of an event or person?

Sometimes i feel sad that i can't listen to certain favourite songs or bands because they remind me too much of ex partners. Sometimes i try and it just brings up a bunch of memories I'd rather not think about.
Are these songs lost to me forever? Has anyone had this and found that with enough time passed they can love the music again without it being tainted?