how should you respond if a situation like this occurs

if someone wrongly associates you with something or people you have nothing to do with, you are nothing to do with those people, yet someone still associates you with them?

how should you respond to that situation without losing your temper?