Actually our "two-party" system has been 4-5 parties most of my lifetime. I say 5, because FDR and the New Deal sort of broadened the Democratic tent big enough for the full spectrum, or to more precise, his efforts to respond to the Great Depression and WWII when the Republicans essentially wanted to do nothing (prior to Pearl Harbor). Truman's efforts to steer a mid-course in the post-war period (integrating the military, supporting the UN that alienated the conservatives; temporarily nationalizing strategic industries to fight the unions, price controls alienating the conservatives) actually split the Democratic Party into 3 parties: Wallace leading the liberal/progressives in one direction, Thurmand the conservative/States' Righter Dixiecrats in the other, and assuring Republican Dewey's victory, the pundits all assured us. Except Truman rode the moderate Democratic ticket to victory.
The Republicans then recruited Eisenhower to be the moderate middleroader of all time, and road him to victory in two terms. And during the 50's the two parties tried to outdo each other in being middle of the roaders, working across the aisle; Adlai Stevenson was just a little too liberal to fit the mold on a national scale. The liberals and conservatives drifted back into one or the other of the two parties, but were always fringes within the parties driven by the moderates.
It was JFK/LBJ's embrace of the Civil Rights Movement that drove a more permanent wedge between the two parties, and Nixon's Southern Strategy that converted the long time Democratic South into rock solid conservative Republicans. And it seems that is when the two parties became taken over by the extreme views in both parties, leaving the moderates no place to go.
All of which is a long-winded explanation as to why there should be no limit, but at least 3 is needed.