People find it hard to spend time with me

I kinda noticed that like 75% of the people don't feel well in my presence, they cut the conversation short, want to listen to music when waiting I'm the only person there, get upset very easily like I'd trigger something in them and generally find it hard to be with me, sometimes I can see how they are relieved when a third person joins, so that they are not with me anymore. I've never known why but it's a repeating pattern, so it's on me.

There is one person I would like to be friends with but I kinda feel like I should stop wanting to be friends with them and leave them alone. They didn't say anything but I don't want to make their life hard by them having to spend time with me in the long run.