Just because a set is the nuts one one board doesn't mean it's the nuts on another board. We have a good hand capable of getting value from lots of hands but we don't have the nuts.

We don't stack off with our set of aces when the board runs out with a 4 flush. Some cards are bad for us, most aren't. If no cards were bad for us we'd win every hand. So I think you need to stop viewing it like you are and think about your range.

If the turn is an 8 or Q that sucks balls and we can probably find folds as our range contains all the more nutty stuff but people are worried about these cards too and we have position and a 3 flush isn't the end of the world. If you're shutting down every time there's a possible flush then you're doing something wrong especially when he doesn't have that many flushes in his range.