I would have definately called, his all-in was only around 1/2 the pot. He probably had a jack and figured he'd try to push you off.