Why is “ we just grew apart” or “because I wanted to” not a good enough reason for divorce it seems?

Seems anytime I mention being divorced, it's automatically assumed he cheated on me or something equally juicy.
Then they are disappointed when I say that we grew apart.
Like it has to be something dramatic.