I prefer to leave the shoes at home when practical, but this is just a matter of choice. I can understand some places may not be appropriate for bare feet, but they are few. I don't get the anger if someone goes into a convenience store to purchase a soft drink and a bag of chips without shoes. How is that hurting anyone? If another person's choice does not affect you, it should not bother you.
