[media=https://www.youtube.com/watch?v=vgGEZ_uqt9M]
"Legacy" Season 4, Episode 6, Star Trek the Next Generation
I sense he values you very much and has stayed with you for this reason and in hindsight probably regrets betraying you. He may have felt that the rules were different back then, but over time has come to realize its impact in that he may lose you and so has not told you for fear of that possibility.
But that's not his decision to make, it's yours. One thing is for sure, the moment you eventually have him look you square in the face and tell you the truth, that will be the pivotal moment. You will have your answer in full right there.
One thing I've noticed in relationships is that the stronger ones let it go, leave it at that and move on with their lives. The rest may not, and that's ok. Everyone is different and in different stages of life. It's a very personal decision weighing the consequences for the future in light of things in the past. However like a tree, the future is constantly growing and changing, while the past is like a ghost, dead and can never change.
PS - See the episode above to find out what the tiny amber object meant. Hope this helps you.
