I think I'd only be worried on the river if he shoved over my raise. And even then I think I'd find trip aces hard to fold. What did he have?