Your mental health is more important than your familys relationship with him. If their potential pain is all thats holding you back then I think theres no excuse not to tell them. I'm not going to judge you for whatever you decide but I think that if you have the will to tell them what happened and you think they'll believe you then there's no reason you shouldn't.