9 research outputs found
The Sensitivity of Counterfactual Fairness to Unmeasured Confounding
Causal approaches to fairness have seen substantial recent interest, both
from the machine learning community and from wider parties interested in
ethical prediction algorithms. In no small part, this has been due to the fact
that causal models allow one to simultaneously leverage data and expert
knowledge to remove discriminatory effects from predictions. However, one of
the primary assumptions in causal modeling is that you know the causal graph.
This introduces a new opportunity for bias, caused by misspecifying the causal
model. One common way for misspecification to occur is via unmeasured
confounding: the true causal effect between variables is partially described by
unobserved quantities. In this work we design tools to assess the sensitivity
of fairness measures to this confounding for the popular class of non-linear
additive noise models (ANMs). Specifically, we give a procedure for computing
the maximum difference between two counterfactually fair predictors, where one
has become biased due to confounding. For the case of bivariate confounding our
technique can be swiftly computed via a sequence of closed-form updates. For
multivariate confounding we give an algorithm that can be efficiently solved
via automatic differentiation. We demonstrate our new sensitivity analysis
tools in real-world fairness scenarios to assess the bias arising from
confounding.Comment: published at UAI 201
The Sensitivity of Counterfactual Fairness to Unmeasured Confounding
Causal approaches to fairness have seen substantial recent interest, both from the machine
learning community and from wider parties interested in ethical prediction algorithms. In
no small part, this has been due to the fact
that causal models allow one to simultaneously
leverage data and expert knowledge to remove
discriminatory effects from predictions. However, one of the primary assumptions in causal
modeling is that you know the causal graph.
This introduces a new opportunity for bias,
caused by misspecifying the causal model.
One common way for misspecification to occur is via unmeasured confounding: the true
causal effect between variables is partially described by unobserved quantities. In this work
we design tools to assess the sensitivity of fairness measures to this confounding for the popular class of non-linear additive noise models (ANMs). Specifically, we give a procedure for computing the maximum difference
between two counterfactually fair predictors,
where one has become biased due to confounding. For the case of bivariate confounding our
technique can be swiftly computed via a sequence of closed-form updates. For multivariate confounding we give an algorithm that can
be efficiently solved via automatic differentiation. We demonstrate our new sensitivity analysis tools in real-world fairness scenarios to assess the bias arising from confounding
Approaches to Improve Fairness when Deploying AI-based Algorithms in Hiring – Using a Systematic Literature Review to Guide Future Research
Algorithmic fairness in Information Systems (IS) is a concept that aims to mitigate systematic discrimination and bias in automated decision making. However, previous research argued that different fairness criteria are often incompatible. In hiring, AI is used to assess and rank applicants according to their fit for vacant positions. However, various types of bias also exist for AI-based algorithms (e.g., using biased historical data). To reduce AI’s bias and thereby unfair treatment, we conducted a systematic literature review to identify suitable strategies for the context of hiring. We identified nine fundamental articles in this context and extracted four types of approaches to address unfairness in AI, namely pre-process, in-process, post-process, and feature selection. Based on our findings, we (a) derived a research agenda for future studies and (b) proposed strategies for practitioners who design and develop AIs for hiring purposes
Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics
Abstract: There is growing concern that decision-making informed by machine learning (ML) algorithms may unfairly discriminate based on personal demographic attributes, such as race and gender. Scholars have responded by introducing numerous mathematical definitions of fairness to test the algorithm, many of which are in conflict with one another. However, these reductionist representations of fairness often bear little resemblance to real-life fairness considerations, which in practice are highly contextual. Moreover, fairness metrics tend to be implemented within narrow and targeted fairness toolkits for algorithm assessments that are difficult to integrate into an algorithm’s broader ethical assessment. In this paper, we derive lessons from ethical philosophy and welfare economics as they relate to the contextual factors relevant for fairness. In particular we highlight the debate around the acceptability of particular inequalities and the inextricable links between fairness, welfare and autonomy. We propose Key Ethics Indicators (KEIs) as a way towards providing a more holistic understanding of whether or not an algorithm is aligned to the decision-maker’s ethical values
Partial Counterfactual Identification of Continuous Outcomes with a Curvature Sensitivity Model
Counterfactual inference aims to answer retrospective ''what if'' questions
and thus belongs to the most fine-grained type of inference in Pearl's
causality ladder. Existing methods for counterfactual inference with continuous
outcomes aim at point identification and thus make strong and unnatural
assumptions about the underlying structural causal model. In this paper, we
relax these assumptions and aim at partial counterfactual identification of
continuous outcomes, i.e., when the counterfactual query resides in an
ignorance interval with informative bounds. We prove that, in general, the
ignorance interval of the counterfactual queries has non-informative bounds,
already when functions of structural causal models are continuously
differentiable. As a remedy, we propose a novel sensitivity model called
Curvature Sensitivity Model. This allows us to obtain informative bounds by
bounding the curvature of level sets of the functions. We further show that
existing point counterfactual identification methods are special cases of our
Curvature Sensitivity Model when the bound of the curvature is set to zero. We
then propose an implementation of our Curvature Sensitivity Model in the form
of a novel deep generative model, which we call Augmented Pseudo-Invertible
Decoder. Our implementation employs (i) residual normalizing flows with (ii)
variational augmentations. We empirically demonstrate the effectiveness of our
Augmented Pseudo-Invertible Decoder. To the best of our knowledge, ours is the
first partial identification model for Markovian structural causal models with
continuous outcomes
The Sensitivity of Counterfactual Fairness to Unmeasured Confounding
Causal approaches to fairness have seen substantial recent interest, both from the machine learning community and from wider parties interested in ethical prediction algorithms. In no small part, this has been due to the fact that causal models allow one to simultaneously leverage data and expert knowledge to remove discriminatory effects from predictions. However, one of the primary assumptions in causal modeling is that you know the causal graph. This introduces a new opportunity for bias, caused by misspecifying the causal model. One common way for misspecification to occur is via unmeasured confounding: the true causal effect between variables is partially described by unobserved quantities. In this work we design tools to assess the sensitivity of fairness measures to this confounding for the popular class of non-linear additive noise models (ANMs). Specifically, we give a procedure for computing the maximum difference between two counterfactually fair predictors, where one has become biased due to confounding. For the case of bivariate confounding our technique can be swiftly computed via a sequence of closed-form updates. For multivariate confounding we give an algorithm that can be efficiently solved via automatic differentiation. We demonstrate our new sensitivity analysis tools in real-world fairness scenarios to assess the bias arising from confounding