Have you given a thought to seek counseling?
Intimacy is an integral part of a healthy marriage and if that important part is missing, then frustration is taking the upper hand. It should not be.
I do commend you on not having taken the path of cheating. It shows strength of character and not everyone does have it. I wish you well