I agree with pretty much all of that. But that isn't what took him down (though it did hurt him significantly). What took him down was a lie. When they take him down like that, what are they going to do to the next guy they don't like? What are they going to do to you or to me?
The amount of power that mere accusations of sexual assault, ones that are even proven as fake, has in our politics and in our society at large is a VERY bad thing.