That idolatry spread to other nations after the Tower of Babel. God divided the nations according to the "sons of God", and they were supposed to shepherd them and report to God, and God would have more DIRECT charge over Israel. However, these sons of God decided, "nah, I would rather have the people worship me". So idolatry got it roots in the worship of fallen angels. Granted before the flood and before the tower of Babel incident the worship of lesser entities other than God existed, but I think it really took root after the nations were divided up after the Tower of Babel incident.