So, if I were to get a horse drawn carriage, would I still need a drivers license?

Before automobiles, no licensing was required to drive a horse (or mule, or donkey, or ox), certainly not in rural areas; in urban areas, people who used horses as an adjunct to their business— e.g., people who sold things like produce (fruits and veggies), milk, firewood or whatever, and had a horse or horses to haul it— may have had to have a business license to operate in the urban boundaries. But they didn’t need a specific license to drive a horse or other animal.

The exception to this was drivers of horse-drawn cabs, who by Victorian times were regulated by laws that specified things like hours the cab driver could work, fares, etc.

Cab drivers were required to be licensed, but I don’t know what, if any, tests they had to pass, or if they just paid a fee.