Most "Black Affinity Housing" takes place on private college campuses. And it's not particularly prevalent. I think there's a few public universities experimenting with it. Far less common for Hispanics and Asians. And while I'll take your statement about white students at face value, even though it smells a little bit like "poor me white boy" perspective, there's already colleges, typically religious, that are entirely white. Just like there's colleges that are all black. They're called HCBU's. But moreover, I think it's safe to say that the vast majority of younger people in college aren't there looking to be segregated. That's for the people who don't go to college. Blacks may be more likely because historically it's been harder for many of them to navigate and there's strength in numbers to get through it.
Personally, I'm 100 percent opposed to segregated housing, irrespective of race, because society isn't segregated. You learn about life in college and "us vs. them" is not a good jumping off point.