If he hasn't tried before this, and from the sounds of it he hasn't, then honestly your life is yours, do with it what you want. If YOU want him there, then so be it, otherwise, you don't even have to answer him. If you choose to answer him, then you can bring up the fact that he hasn't seemed to care enough the last five years so he gave up any rights for the present and future.
