What's wrong with polygamy?

People can get really bent out of shape about polygamy. As if it's some kind of moral crime. It's even illegal in developed countries.
But what is actually wrong with it between consenting adults?