Being easygoing is terrible.

Not because people will take advantage of you, I can live with that. It's terrible because most other people are a lot harsher, and will call you out on minor mistakes. So if you have any kind of friendship or relationship with someone else, the both of you are gonna make perceived mistakes from time to time. You just are, it's inevitable. When they make 'em, you won't care, so they won't notice. When you make them, they will care, so you will notice. And that leads to an increasingly bad imbalance where your problems just build up and up and up while they stay squeaky clean, and you end up looking like the bad guy -- the one who's always apologizing.

There's no recourse for this either, unless you're willing to feign being offended when you're not. It's just a natural consequence of the differences between your social standards and theirs.