@BabyLonia Yes, the only way I can remove these people whom I do not follow and so I am already "unfriended" from that list--by clicking "Remove"--shows exactly what I said: you use a follower block; see:
@BabyLonia Not sure that would change the fact that they followed you, but perhaps; considering the long-lived nature of these lists, it would be odd if those maneuvers could change the other user's decision to follow you....
@BabyLonia Answer...yes, one CAN remove them (adding to follower blocks), then UNblock them...and they do NOT come back.
By getting rid of all the SW-User Xs and banned users at least one year old, the list of "followers of me whom I don't follow back" went from >3 pp (but under 4) to under 2 pp, and the entries aren't riddled with that nonsense.