So I would really like to stop wearing bras for my own personal comfort but....

You could wear a burlap sack and it's still going to happen. I would imagine the trick is to just come to the understanding that there's weak, desperate, selfish dudes out there. There's not much you can do about that. Write them off and just wear what your comfortable with. They're tits. Who cares?

But I would say, as was mentioned below, your situation is being dictated by women who dress SO they can get attention from men. Equally screwed up from where I'm standing.