I did it all the time. I never had actual swimwear. When me and my friends use to pass this lake in the woods after school on hot days one friend had the idea of swimming in his underwear. Since I didn't have underwear either i decided to swim naked. Once I was totally naked the rest of my friends followed.