Operationalizing Individual Fairness with Pairwise Fair Representations

We revisit the notion of individual fairness proposed by Dwork et al. A central challenge in operationalizing their approach is the difficulty in eliciting a human specification of a similarity metric. In this paper, we propose an operationalization of individual fairness that does not rely on a human specification of a distance metric. Instead, we propose novel approaches to elicit and leverage side-information on equally deserving individuals to counter subordination between social groups. We model this knowledge as a fairness graph, and learn a unified Pairwise Fair Representation (PFR) of the data that captures both data-driven similarity between individuals and the pairwise side-information in fairness graph. We elicit fairness judgments from a variety of sources, including human judgments for two real-world datasets on recidivism prediction (COMPAS) and violent neighborhood prediction (Crime & Communities). Our experiments show that the PFR model for operationalizing individual fairness is practically viable.Comment: To be published in the proceedings of the VLDB Endowment, Vol. 13, Issue.

{iFair}: {L}earning Individually Fair Data Representations for Algorithmic Decision Making

People are rated and ranked, towards algorithmic decision making in an increasing number of applications, typically based on machine learning. Research on how to incorporate fairness into such tasks has prevalently pursued the paradigm of group fairness: ensuring that each ethnic or social group receives its fair share in the outcome of classifiers and rankings. In contrast, the alternative paradigm of individual fairness has received relatively little attention. This paper introduces a method for probabilistically clustering user records into a low-rank representation that captures individual fairness yet also achieves high accuracy in classification and regression models. Our notion of individual fairness requires that users who are similar in all task-relevant attributes such as job qualification, and disregarding all potentially discriminating attributes such as gender, should have similar outcomes. Since the case for fairness is ubiquitous across many tasks, we aim to learn general representations that can be applied to arbitrary downstream use-cases. We demonstrate the versatility of our method by applying it to classification and learning-to-rank tasks on two real-world datasets. Our experiments show substantial improvements over the best prior work for this setting

iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making

People are rated and ranked, towards algorithmic decision making in an increasing number of applications, typically based on machine learning. Research on how to incorporate fairness into such tasks has prevalently pursued the paradigm of group fairness: giving adequate success rates to specifically protected groups. In contrast, the alternative paradigm of individual fairness has received relatively little attention, and this paper advances this less explored direction. The paper introduces a method for probabilistically mapping user records into a low-rank representation that reconciles individual fairness and the utility of classifiers and rankings in downstream applications. Our notion of individual fairness requires that users who are similar in all task-relevant attributes such as job qualification, and disregarding all potentially discriminating attributes such as gender, should have similar outcomes. We demonstrate the versatility of our method by applying it to classification and learning-to-rank tasks on a variety of real-world datasets. Our experiments show substantial improvements over the best prior work for this setting.Comment: Accepted at ICDE 2019. Please cite the ICDE 2019 proceedings versio

Operationalizing Individual Fairness with Pairwise Fair Representations

We revisit the notion of individual fairness proposed by Dwork et al. A central challenge in operationalizing their approach is the difficulty in eliciting a human specification of a similarity metric. In this paper, we propose an operationalization of individual fairness that does not rely on a human specification of a distance metric. Instead, we propose novel approaches to elicit and leverage side-information on equally deserving individuals to counter subordination between social groups. We model this knowledge as a fairness graph, and learn a unified Pairwise Fair Representation(PFR) of the data that captures both data-driven similarity between individuals and the pairwise side-information in fairness graph. We elicit fairness judgments from a variety of sources, including humans judgments for two real-world datasets on recidivism prediction (COMPAS) and violent neighborhood prediction (Crime & Communities). Our experiments show that the PFR model for operationalizing individual fairness is practically viable

Detecting and Mitigating Test-time Failure Risks via Model-agnostic Uncertainty Learning

Reliably predicting potential failure risks of machine learning (ML) systems when deployed with production data is a crucial aspect of trustworthy AI. This paper introduces Risk Advisor, a novel post-hoc meta-learner for estimating failure risks and predictive uncertainties of any already-trained black-box classification model. In addition to providing a risk score, the Risk Advisor decomposes the uncertainty estimates into aleatoric and epistemic uncertainty components, thus giving informative insights into the sources of uncertainty inducing the failures. Consequently, Risk Advisor can distinguish between failures caused by data variability, data shifts and model limitations and advise on mitigation actions (e.g., collecting more data to counter data shift). Extensive experiments on various families of black-box classification models and on real-world and synthetic datasets covering common ML failure scenarios show that the Risk Advisor reliably predicts deployment-time failure risks in all the scenarios, and outperforms strong baselines

Endocrine Dysfunction in Diamond-Blackfan Anemia (DBA): A Report from the DBA Registry (DBAR)

BACKGROUND: Diamond-Blackfan anemia (DBA) is a rare inherited bone marrow failure syndrome. The mainstays of treatment involve chronic red cell transfusions, long-term glucocorticoid therapy, and stem cell transplantation. Systematic data concerning endocrine function in DBA are limited. We studied patients in the DBA Registry (DBAR) of North America to assess the prevalence of various endocrinopathies. PROCEDURE: In a pilot study, retrospective data were collected for 12 patients with DBA. Subsequently, patients with DBA aged 1-39 years were recruited prospectively. Combined, 57 patients were studied; 38 chronically transfused, 12 glucocorticoid-dependent, and seven in remission. Data were collected on anthropometric measurements, systematic screening of pituitary, thyroid, parathyroid, adrenal, pancreatic, and gonadal function, and ferritin levels. Descriptive statistics were tabulated and group differences were assessed. RESULTS: Fifty-three percent of patients had \u3e/=1 endocrine disorder, including adrenal insufficiency (32%), hypogonadism (29%), hypothyroidism (14%), growth hormone dysfunction (7%), diabetes mellitus (2%), and/or diabetes insipidus (2%). Ten of the 33 patients with available heights had height standard deviation less than -2. Low 25-hydroxy vitamin D (25(OH)D) levels were present in 50%. A small proportion also had osteopenia, osteoporosis, or hypercalciuria. Most with adrenal insufficiency were glucocorticoid dependent; other endocrinopathies were more common in chronically transfused patients. CONCLUSIONS: Endocrine dysfunction is common in DBA, as early as the teenage years. Although prevalence is highest in transfused patients, patients taking glucocorticoids or in remission also have endocrine dysfunction. Longitudinal studies are needed to better understand the etiology and true prevalence of these disorders