When cleaning pots/pans after cooking, do you ever fear leaving scratches when scrubbing dirt off?

I get a little paranoid about my cast iron but even if I do scratch the patina it can be rebuilt. Otherwise, nothing lasts forever so I'm just as gentle as I can be with the other stiff and don't stress too much.