How did you handle your kid's bully?

My son WAS the bully. As parents, we would take him to the house of the child he had bullied, or whose toy he had stolen, and we would make him apologize and return the toy.