Should transgenders be allowed to use public washrooms of the gender they identify as?

People worry about hurting transgenders. I think about what women think. I'm a crossdresser and I think women would probably be very uncomfortable with it. They say it's not fair to transgender people. Well it's not fair to women either.