1 research outputs found
When Mitigating Bias is Unfair: A Comprehensive Study on the Impact of Bias Mitigation Algorithms
Most works on the fairness of machine learning systems focus on the blind
optimization of common fairness metrics, such as Demographic Parity and
Equalized Odds. In this paper, we conduct a comparative study of several bias
mitigation approaches to investigate their behaviors at a fine grain, the
prediction level. Our objective is to characterize the differences between fair
models obtained with different approaches. With comparable performances in
fairness and accuracy, are the different bias mitigation approaches impacting a
similar number of individuals? Do they mitigate bias in a similar way? Do they
affect the same individuals when debiasing a model? Our findings show that bias
mitigation approaches differ a lot in their strategies, both in the number of
impacted individuals and the populations targeted. More surprisingly, we show
these results even apply for several runs of the same mitigation approach.
These findings raise questions about the limitations of the current group
fairness metrics, as well as the arbitrariness, hence unfairness, of the whole
debiasing process