Today a colleague offered to drop me home from work because she lives nearby

It gave me so much anxiety. If she gets to know where I live she will see how messy our house is. Because of partially my parents' mental illnesses including compulsive hoarding, and partially the violent neighborhood that makes it unsafe to do any home improvements, we had to leave renovation halfway because our neighbours are convicted criminals and actively started invading our personal space a few years back and we felt threatened. Now there's rubble scattered in the front yard. My parents' marriage was falling apart so my mother refused to invest in home improvements, because the house is my dad's. (laws are different here.)
I wouldn't be able to call her inside because we are barely living in a sane household and she would know how dysfunctional we are.
I didn't know how to refuse. She was insistent and thought that I was just being polite by saying no. Thankfully I got free from work very late and she had to leave.
Now I'm sweating thinking about the next time I'll have to deal with this. Aaagghh.