He does have a point. There are a lot of people who think it's literally impossible to be racist to a white person, which is beautifully ironic, since it's usually left wing people who despise racism, and they are being racist without even realising it.