Give yourself time to think, dont force yourself to go near him if you dont want to or that'll just make you chicken out!! (From experience..) think about how you feel and your thoughts about him, See if your gay, bi, pan, or straight or anything! I believe in u ray :3!

You're trying to date him, not his family.

While it's easier if the family accepts you, there's a good chance that he and his family might be parting ways if he's attracted to you anyway.

If he's not attracted to you, it could be that you are worrying about nothing. Either way, there's other things to worry about before you go worrying about his family.

Take your ex's advice. You'll never know unless you try.

When I was young, I considered myself straight but had a guy friend who I liked hanging out with.
I started hanging out with him more when I was "between girlfriends"
One night when we were at his place drinking and watching sports on TV, he left the room. When he came back a few minutes later he was naked. He walked up to me and kissed me and I felt surprisingly aroused. One thing led to another, and we had sex. I woke up naked in his arms the next morning
The moment he kissed me, I knew I wasn't straight, and I've now determined that I'm gay. Sometimes you don't realize what your true sexuality is until you take the time and effort to figure it out.

Consider this possibilty:

You just enjoy sharing your sexuality with others who enjoy sharing their sexuality.

That's it.