I am a member of a 12-step group called Sex Addicts Anonymous. In my case, my addictive behaviors were mainly on the internet, but there are plenty of members of my meeting who were having multiple affairs before they got sober.
My advice is this: sex in a relationship, or even just for pleasure, is one thing. But sex to fill a void, sex to numb the pain of life, is something else entirely. The latter can become an addictive drug. And I know lots of people who are in AA and SAA, and they all say that sex addiction is by far the harder of the two to overcome.
The way I get through life is to use the extra time I have (as a result of not spending hours online in pursuit of elusive and unreliable satisfaction) to pursue things that refresh my spirit. In my case, reading, walking in the woods, star gazing, listening to music, prayer, journaling, meaningful online conversations, breakfast with friends, are some of the things that help me.
But, there is more to the mystery: after adopting a more sober attitude, a lot of those things in life that caused stress become easier to address. My relationship with my wife improved, my job became less stressful, my self-loathing slowly lifted. The 12 steps, which I assume you are familiar with, are not just about abstinence. They are, as one member of my group (who is a member of AA, NA, and SAA) puts it, about learning to "living life on life's terms." And a lot of that is the ability to be more connected with other people.
I know what it feels like to think it is impossible to face life without some sort of addiction to escape to. I get what you are saying.
But I also know what it is like to get beyond that.