It can potentially put me in a bad turn/river spot if the pot gets bloated when I am oop... folding the best hand is something I hate doing.

Additionally, it is very tough for him to have a hand he can call more than 1-2 bets with. There are still two streets to get value from later so why screw around with getting outplayed oop.