Husband took maternity leave money

So i saved and saved for maternity leave starting 6 months before i took off.. that money was supposed to cover things like mortgage, food, diapers, formula etc, daycare. there was hardly anything extra.. i asked my husband before hand if he would save as well.. we’ll he didn’t .. so went on leave.. halfway through it he comes to me saying he mismanaged his money and needed to borrow 3000 dollars.. turns out he was behind on student loans and a car payment .. he seemed desperate so i agreed.. we’ll i also let him know this was a lot of my money ..now this has put me in a situation where i need to go back early from maternity leave by a month (so 12 weeks off has turned into 8) because i simply won’t be able to safely cover all bills unless he pays me back .. we’ll i asked him to come up with a plan to put that money back (hopefully before i have to go back) and he’s acting like I’m a horrible person for asking.. he shut down the 2 times I’ve brought it up.. I’m seriously just trying to figure out if i truly will have to cut my leave short.. if he can come up with a payment plan i won’t have to go back early .. am i right to be asking for this back and to be upset?