"If you don't want that people misuse your goodness, then you should not be afraid their bad opinion about you"

Yes, you have to be assertive, stand up for yourself and negotiate for what you want. Dont aim to be nice or a people pleaser. But if you have to, be selfish for a while to build up enough power that you can really help people.