Why it's hard to get rid of my Southern accent?

Many years ago I met a fellow who spoke English with a real Canadian accent. He told me that he was an American deserter who had left the US rather than go to war in Vietnam. I didn't put much stock in his story until one day he phoned his Mom in Alabama. I had trouble understanding what he said his southern accent was so thick. Another fellow I knew spoke with a real Scottish burr. He laughed at himself saying that he thought he had a Canadian accent but when he heard himself on a tape recording he thought he had just got off the boat. He then went on to say that when he went back to Scotland to visit all of his friends thought he had an American accent and called him a Yankee.