This is a call. Like OP says, it's only bad if villain does this with exclusively sets and straights, but he is surely wider. I'd say villain can also do this with JJ+, maybe 99/TT too, and probably AKss and most other NFD he has. With these hands in his range, we're in much better shape. And even if he does only have sets and straights, we're in reasonable shape. Yes it's -EV against that range, but even if that is his range, we can take comfort from the fact he played his range in such a way that we don't pay him off with the vast majority of our range. There is no range he can have here that makes his play optimal.

Solid analysis.