Wooden Bench varnish
 

I have a wooden bench that sits on the porch during summer. It has faded a bit from sun and rain. I was thinking about varnishing it and putting a clear water sealer on it like one would put on a deck. My question is, would that rub off on clothing, or does that sound like the way to go?

You guessed it, it will eventually rub off on your clothing.
I would recommend Marine Varnish. It's intended for outdoor use and won't rub off.

I second the vote for marine varnish (also called "Spar Varnish").