The ability to do abstract math with numbers stems from our ability to name those numbers

The human ability for language is the foundation for our ability to reason about numbers in an abstract way.