The government when it comes to polygamy:

Marriage in most western societies confers significant legal privileges regarding property, inheritance, immigration, tax, and general finance. Extending these privileges to a potentially infinite number of spouses would make a whole raft of law impossible to enforce and be open to general fraud and abuse. That is why government is right to intervene.