You can only feel betrayed by a person that you had some type of feelings for, agree?

I suppose that's one way of putting it. I tend to think you can only be betrayed by someone if you trusted them to some extent.