The Church of Rationality lists a few possible reasons why people so often express this odd view. My favorite point:
You're not much of a punk rock insider if you like the Ramones. Everyone likes the Ramones. If you're an expert concerning some Portugese band whose whole output consists of flexidiscs published with Greek fanzines in the late 1980s, then we're talking. Likewise, you're not much of an anti-racist if you're simply against racism. Even the conservatives are against racism these days. You'll have to offer a little more. The more extreme, the better.