You are not bankrolled for the $3 and are definitely not for the $4. If you can deposit another $100 and are having fun then it's fine, but if this is your only $100 you need to play lower buy-ins.

The $3 are pretty easy, you should probably be running better than breakeven over 100. I doubt it is all variance.

$32/0.05 = 640bb so 0.4 bb/100. Not so great.

Read all the stickies here. Try the $1-45 man or $1.5-18 man turbos until you are confident you are a winning player (or the 5NL, are you more confident with SnG or ring). Don't play the $4-180 man.