It would be too many when there aren't enough resources to support the population.

Right now barriers to resources are man-made.

Get rid of the elites that are hoarding land and the problem will work itself out.