Yeah, the strongest hand you can put this guy on is 6-6. He would've reraised preflop with K-K or Q-Q, and he would have to have the only two remaining kings or queens in the deck if he were to have either of these hands. Essentially, you are only losing to one remotely likely hand, while you're probably up against a couple other strong hands that are almost drawing dead. I would move all-in.

I actually just got involved in a hand that was almost identical to this, except the other guy had top two pair, and I actually did have bottom set.