Influence of Label and Selection Bias on Fairness Interventions

Bias can be introduced in different ways in machine learning datasets, with the bias type influencing the effectiveness of fairness interventions. In this work, we model fair worlds and their biased counterparts by introducing controlled label and selection bias in real-life datasets with low discrimination. We then analyze the resulting prediction models, with or without bias mitigation. Our results provide some guidance on the use of reweighing, massaging and Fairness Through Unawareness, and show that other dataset characteristics also play a role on fairness intervention efficiency, calling for further research.