Would you be annoyed?

Whenever you are on the phone with someone and they get a call, they take that call and say they'll call you back. And this can happen several times a day, EVERYTIME you're on the phone with them.

Or are you the person that rejects the current call for the incoming call?