Because institutionalized racism is a really terrible thing. It certainly was something that I was brought up with in south Louisiana. I remember my father telling me that "they are not like us. You can be friendly, but you can't be friends with them," as if black people couldn't relate to me or would turn on me because I was white. I'm definitely one of these left leaning people you are talking about, and it is partly because of the disparity and the racism that I witnessed growing up. The neighborhoods and the schools were segregated, and while racism was rampant on both sides of the street, I completely understand why someone might be a little angry that they were not afforded the same opportunities for education or jobs. Stating that racism is not just a white problem does not negate the fact that institutionalized racism exists, so for me, that is not a valid argument. Yes, there are other groups who discriminate and who are discriminated against, but for a very long time whites males held social power and still do.
