What's the 'trick' part of 'trick or treat' for Halloween meant to be about?

When we were kids, we only played tricks on those grumpy families who didn't give treats. The tricks were usually something like soaping car windows, hiding porch furniture, or putting dog poop in front of their door. We never damaged anything.