How do I deal with this situation?

Me and my boyfriend have been together for 3 months now. I have high anxiety and suffer a lot of panic attacks-especially at night. My boyfriend has become a big aid with that. The other day we got into an argument and he mentioned that his friends (old coworkers that became good friends) think I’m a handful. I’ve never really met them or hung out with them so they obviously only have an opinion based on what my boyfriend tells them. He sees them multiple times a week-at least once a week. I asked him why they’d think that and that it makes me unhappy to be seen like that by strangers. He said that maybe one day we could all hang out and I could show them what I’m like. I agreed. So today, I asked to see him later but he told me he already had plans with his friends tonight at 8 and that they’re going to an arcade. I was hoping he’d invite me but he never did. It’s been 3 hours since he told me he was on his way to meet them. im just feeling really neglected. He’s our with people who don’t have a good opinion about me and I wasn’t invited once again. He never takes me out to anything that involves other people. He occasionally takes me out on dates but just us two. I feel like he’s not proud to be with me. He doesn’t post or talk about me on social media. I know not everyone does that but it just aids in the whole pride thing. I don’t know how to bring any of this up or if I even should.