By saying to a man "you can't use the women's showers" you are not taking away human rights from the man. That's ridiculous. Why doesn't the same apply for trans people?