Why is Polyamory so looked down on by society?

Is it that much of a problem to love multiple people so deeply in their eyes?