What is it like to start a big family at 40 years old for man do you think it’s too late?

The older you are - the higher risk of birth defects:

Sure it's true a man can father a child in their later years, but that that comes at a risk as well