Have we grown too far apart ?

When my husband and I first met, it was the time of my life. We were so in love.. he was so full of wisdom, charm, smarts, I felt lucky to even be standing 10 feet away from him. He was like this totally rockstar to me.. fast forward 6 years..I’m in this weird place in life. Had a baby a year ago, he and I BOTH have lost our full time jobs. I’m really lucky though, I’ve started my own business and am paying the $2k a month mortgage with my income. Husband, previously an attorney has been unemployed for a year, hasn’t even tried to look for a job. We’ve been together for 7. The more I dive into this, the more I realize he doesn’t want to work. He even admits it.. he would rather be at home taking care of the baby.He’s spent 60k in savings over the last year on beer and baby toys mostly, going out to eat, Frivolously blown it. All while I was trying to build our credit to by a house... come to find out all of a sudden “he has $800” left in his account ..I’m shocked...I offered for him to start working for me but truth is, he’s lazy as hell. I ask him to do things, he gives excuses why he can’t.. by end of day he’s don’t nothing...He won’t even put his resume together. I feel like we have grown apart - there is zero affection, he doesn’t want to do “it” ever for months... because he insists the baby sleeps with us.. I’m just at a loss. Feel like I’m just supporting him when he has no desire to make his life better.. by making sure our rent is paid I’m enabling him... but I have to to provide a roof over our heads. Its a conundrum..It’d be different if there was love and affection but there isn’t and hasn’t been.. for a long long time.. and although I’ve told myself I would NEVER leave him, and meant it.. for the first time I’m starting to realize he’s really dragging me down. He’s mad that I work so much, I left my job at a #3 tech company because “I was working to many hours” for his liking... I’m in this place where for the first time I’m realizing he brings zero of anything (except taking care of the baby) to the table.. he’s told me he’s afraid I’m going to leave.. and doesn’t want to live at the thought of that.. and I always chime back I’ll never leave.. but now.. after the dust has settled.. there is this 1% part of my heart that is telling me I might want to think about opening that door... he’s not the man I married and far far far from it.. and there is no affection anymore.. I’m deeply unhappy. But have values in “sticking things out”. Has anyone been here? Did it get better? You know the minute I leave he’s going to get his shit together.. I wish it didn’t take that.. but I don’t know what to do.. please someone tell me your story.. where do I go from here?