My figs are shrivelling up, Does anybody have experience with fig trees?

The fig tree I planted 12 months ago is bearing fruit. There supposed to be the plumb purply brown ones but when the go from green to brown the shrivel up and fall off. I have been using plant feed, fertiliser and mulch and even talk to it. (That's okay, my neighbours already think I'm weird) but nothings working.