"I'm sick of playing with you" comment, kind of makes me want to fold.

I think his range has lots of low/mid pairs. Was he being aggressive? If not, then I can see someone surprising with a strong hand.

I would meh just fold.