He's tearing you apart with his little bets/raises to get you to fold when you miss. If you just check/call down you won't get lots of value from his bets, but if you take over you might get him to make a really bad calldown for lots of money.
Don't confuse this clown with the guys who only bet it when they got it. His typical hand is not much of anything.