i have always had trouble making friends

I have always had a hard time making friends. I only remember 1 or 2 guys when i was in school at a very young age whom i used to call best friends and that's it. Somehow even to this day i can only have 1-2 friends in my life at max. it becomes tough when that 1 or two friend has other friends and is busy and cannot indulge in all the kinds of activities that i indulge in which makes me look like a loner. I would blame the society or the people around me but since this has been happening to me since childhood i am guessing the problem has something to do with me.
maybe i am picky about making friends, maybe these people presented themselves as friends at some point and i refused because i was too cool for them and later i felt like i wanted to be friends with them because i felt lonely.
Can anyone relate or give me advice on this? would appreciate it. thanks!