How would you deal with neighbors filing false noise complaints

I have been living in my current apartment for a year now. I keep to myself, I never had a complaint, but the female neighbor below me seems to have a problem with me based seemingly on the fact that I am different in a way she doesn't like. She immediately got her - and sorry for the English slang - chavvy boyfriend to come up and knock on my door with his hood covering his face telling me "if you keep making noise I will effing flip out", not telling me what the apparent noise was. I had been at my friend's apartment for several days and upon returning home I received a letter from the housing association saying that on the days I had been at my friend's apartment, the neighbors below complained about noise. The housing officer came over to see if she could hear the noise as I'd close all my doors and cupboards loudly. She heard nothing so the female said "oh, erm.. his door closes too loudly at night... i have a baby!" It was still noted as me making antisocial noise, despite there being any evidence

Since then I've been pretty paranoid, trying my hardest to not make any noise whatsoever. Yet a few days ago I receive another letter stating how this neighbor complained that I "played my music from the middle of the noise to the early hours of the morning" and again the complaint is being taken very seriously and the housing officer is coming over again. I worry because I know that associations really bend over backwards for these manipulative people as they know how to manipulate the system, and they will go overly strict with me telling me that I am not allowed to even open my door "loudly".

What should I do?