I feel awkward when someone complained I assumed their gender

I mean I understand not everyone identifies the same way and I’ll call you whatever you want they/them for non binary and gender neutral people and intersex people who don’t identify as male or female, she/her, and he/him. Now I try to keep sir or ma’ am/miss out of my vocabulary in how I address people because I don’t want to make another mistake. I apologized for misgendering someone who looked like a woman and felt awful but again people have eyes and ears and we notice biological sex traits. This person was genderqueer or gender fluid I think. I called them ma’am. Have you ever experienced situations like this?