Do you think it's selfish to ignore people who are negative and perpetually play the victim?

No, but usually they are behaving like that for a reason and don’t mean to be a burden, rather than ignoring them which can feel really hurtful just let them know your boundaries or that you no longer wish to have contact.