Does age REALLY matter

I just can't believe age is still a huge issue when it comes to relationships. The stares I get are so annoying I don't look like a teen so I don't understand... when I mention my fiancé being in his 50s and me having just turned 20 surprises so many people... can't believe it