How do you break up with someone when your kids are attached to them more than you ?

I love him but he’s not my type anymore. He does things that disturbs me. He’s semi nice to the kids. Major turn off!!!! He always compare when one does better than the other. He comes straight in and gets in the bed and doesn’t shower. Street clothes germs is a peeve for me. He’s a junky person and he doesn’t want to get a job and take care of his kids he has the money to help one of his kids for school stuff and complains about it.... how do you break up with someone when he isn’t aware of why without telling him all these bashful things that I’ve expressed before and have told him that I don’t like but wont change. It’s hard breaking up with him I always end up feeling sorry for him because he says how he doesn’t feel comfortable at other people’s house he feels he do a lot when he doesn’t do enough. I’m confused I don’t know what to do 😥