Algorithms for Social Good: A Study of Fairness and Bias in Automated Data-Driven Decision-Making Systems

L'abstract è presente nell'allegato / the abstract is in the attachmen

Operationalizing fairness for responsible machine learning

As machine learning (ML) is increasingly used for decision making in scenarios that impact humans, there is a growing awareness of its potential for unfairness. A large body of recent work has focused on proposing formal notions of fairness in ML, as well as approaches to mitigate unfairness. However, there is a growing disconnect between the ML fairness literature and the needs to operationalize fairness in practice. This thesis addresses the need for responsible ML by developing new models and methods to address challenges in operationalizing fairness in practice. Specifically, it makes the following contributions. First, we tackle a key assumption in the group fairness literature that sensitive demographic attributes such as race and gender are known upfront, and can be readily used in model training to mitigate unfairness. In practice, factors like privacy and regulation often prohibit ML models from collecting or using protected attributes in decision making. To address this challenge we introduce the novel notion of computationally-identifiable errors and propose Adversarially Reweighted Learning (ARL), an optimization method that seeks to improve the worst-case performance over unobserved groups, without requiring access to the protected attributes in the dataset. Second, we argue that while group fairness notions are a desirable fairness criterion, they are fundamentally limited as they reduce fairness to an average statistic over pre-identified protected groups. In practice, automated decisions are made at an individual level, and can adversely impact individual people irrespective of the group statistic. We advance the paradigm of individual fairness by proposing iFair (individually fair representations), an optimization approach for learning a low dimensional latent representation of the data with two goals: to encode the data as well as possible, while removing any information about protected attributes in the transformed representation. Third, we advance the individual fairness paradigm, which requires that similar individuals receive similar outcomes. However, similarity metrics computed over observed feature space can be brittle, and inherently limited in their ability to accurately capture similarity between individuals. To address this, we introduce a novel notion of fairness graphs, wherein pairs of individuals can be identified as deemed similar with respect to the ML objective. We cast the problem of individual fairness into graph embedding, and propose PFR (pairwise fair representations), a method to learn a unified pairwise fair representation of the data. Fourth, we tackle the challenge that production data after model deployment is constantly evolving. As a consequence, in spite of the best efforts in training a fair model, ML systems can be prone to failure risks due to a variety of unforeseen reasons. To ensure responsible model deployment, potential failure risks need to be predicted, and mitigation actions need to be devised, for example, deferring to a human expert when uncertain or collecting additional data to address model’s blind-spots. We propose Risk Advisor, a model-agnostic meta-learner to predict potential failure risks and to give guidance on the sources of uncertainty inducing the risks, by leveraging information theoretic notions of aleatoric and epistemic uncertainty. This dissertation brings ML fairness closer to real-world applications by developing methods that address key practical challenges. Extensive experiments on a variety of real-world and synthetic datasets show that our proposed methods are viable in practice.Mit der zunehmenden Verwendung von Maschinellem Lernen (ML) in Situationen, die Auswirkungen auf Menschen haben, nimmt das Bewusstsein über das Potenzial für Unfair- ness zu. Ein großer Teil der jüngeren Forschung hat den Fokus auf das formale Verständnis von Fairness im Zusammenhang mit ML sowie auf Ansätze zur Überwindung von Unfairness gelegt. Jedoch driften die Literatur zu Fairness in ML und die Anforderungen zur Implementierung in der Praxis zunehmend auseinander. Diese Arbeit beschäftigt sich mit der Notwendigkeit für verantwortungsvolles ML, wofür neue Modelle und Methoden entwickelt werden, um die Herausforderungen im Fairness-Bereich in der Praxis zu bewältigen. Ihr wissenschaftlicher Beitrag ist im Folgenden dargestellt. In Kapitel 3 behandeln wir die Schlüsselprämisse in der Gruppenfairnessliteratur, dass sensible demografische Merkmale wie etwa die ethnische Zugehörigkeit oder das Geschlecht im Vorhinein bekannt sind und während des Trainings eines Modells zur Reduzierung der Unfairness genutzt werden können. In der Praxis hindern häufig Einschränkungen zum Schutz der Privatsphäre oder gesetzliche Regelungen ML-Modelle daran, geschützte Merkmale für die Entscheidungsfindung zu sammeln oder zu verwenden. Um diese Herausforderung zu überwinden, führen wir das Konzept der Komputational-identifizierbaren Fehler ein und stellen Adversarially Reweighted Learning (ARL) vor, ein Optimierungsverfahren, das die Worst-Case-Performance bei unbekannter Gruppenzugehörigkeit ohne Wissen über die geschützten Merkmale verbessert. In Kapitel 4 stellen wir dar, dass Konzepte für Gruppenfairness trotz ihrer Eignung als Fairnesskriterium grundsätzlich beschränkt sind, da Fairness auf eine gemittelte statistische Größe für zuvor identifizierte geschützte Gruppen reduziert wird. In der Praxis werden automatisierte Entscheidungen auf einer individuellen Ebene gefällt, und können unabhängig von der gruppenbezogenen Statistik Nachteile für Individuen haben. Wir erweitern das Konzept der individuellen Fairness um unsere Methode iFair (individually fair representations), ein Optimierungsverfahren zum Erlernen einer niedrigdimensionalen Darstellung der Daten mit zwei Zielen: die Daten so akkurat wie möglich zu enkodieren und gleichzeitig jegliche Information über die geschützten Merkmale in der transformierten Darstellung zu entfernen. In Kapitel 5 entwickeln wir das Paradigma der individuellen Fairness weiter, das ein ähnliches Ergebnis für ähnliche Individuen erfordert. Ähnlichkeitsmetriken im beobachteten Featureraum können jedoch unzuverlässig und inhärent beschränkt darin sein, Ähnlichkeit zwischen Individuen korrekt abzubilden. Um diese Herausforderung anzugehen, führen wir den neue Konzept der Fairnessgraphen ein, in denen Paare (oder Sets) von Individuen als ähnlich im Bezug auf die ML-Aufgabe identifiziert werden. Wir übersetzen das Problem der individuellen Fairness in eine Grapheinbindung und stellen PFR (pairwise fair representations) vor, eine Methode zum Erlernen einer vereinheitlichten paarweisen fairen Abbildung der Daten. In Kapitel 6 gehen wir die Herausforderung an, dass sich die Daten im Feld nach der Inbetriebnahme des Modells fortlaufend ändern. In der Konsequenz können ML-Systeme trotz größter Bemühungen, ein faires Modell zu trainieren, aufgrund einer Vielzahl an unvorhergesehenen Gründen scheitern. Um eine verantwortungsvolle Implementierung sicherzustellen, gilt es, Risiken für ein potenzielles Versagen vorherzusehen und Gegenmaßnahmen zu entwickeln,z.B. die Übertragung der Entscheidung an einen menschlichen Experten bei Unsicherheit oder das Sammeln weiterer Daten, um die blinden Flecken des Modells abzudecken. Wir stellen mit Risk Advisor einen modell-agnostischen Meta-Learner vor, der Risiken für potenzielles Versagen vorhersagt und Anhaltspunkte für die Ursache der zugrundeliegenden Unsicherheit basierend auf informationstheoretischen Konzepten der aleatorischen und epistemischen Unsicherheit liefert. Diese Dissertation bringt Fairness für verantwortungsvolles ML durch die Entwicklung von Ansätzen für die Lösung von praktischen Kernproblemen näher an die Anwendungen im Feld. Umfassende Experimente mit einer Vielzahl von synthetischen und realen Datensätzen zeigen, dass unsere Ansätze in der Praxis umsetzbar sind.The International Max Planck Research School for Computer Science (IMPRS-CS

Explainability and Fairness in Machine Learning

Over the years, when speaking about prediction models, the focus has been set on improving their accuracy, at the cost of loosing any comprehension of how the model predicts. Consequently, it has also been lost the ability of knowing if the behavior of the model is correct. Moreover, due to the fact that the addresses of the predictions do not have information about how ethic or fair the model is when predicting, persons become reticent to use such type of models. Therefore, in the last years there have been developed investigations aiming to explain such predictions in order to make them intelligible for humans, using techniques like LIME or SHAP, responsible for explaining in an interpretable way what happens behind the prediction. This work addresses this issue and reviews recent literature on the topic.A lo largo de los años, en el ámbito de los modelos de predicción, el foco se ha centrado en mejorar las predicciones realizadas por los modelos, perdiendo a cambio toda comprensión a cerca de cómo el modelo realiza la predicción. La pérdida de comprensión conlleva además el desconocimiento del correcto funcionamiento del modelo, así como reticencias a usar dicho modelo de las personas destinatarias de las predicciones al no poseer información acerca de aspectos éticos y justos a la hora de realizar las predicciones. Es por ello que en los últimos años se ha investigado cómo explicar éstas para así hacerlas de nuevo intelegibles para el ser humano, desarrollando técnicas como LIME y SHAP, encargadas de exponer en una forma interpretable por el ser humano lo que sucede detrás de la predicción. En este trabajo abordamos este tema, y revisamos la literatura existente sobre el mismo.Universidad de Sevilla. Grado en Física y Matématica

The evil, poor, disliked and punished: criminal stereotypes and the effects of their cognitive, affective and behavioural outcomes on punitiveness toward crime

Why does the public so staunchly support harsh criminal justice policies when the social, fiscal and political costs are so great? Individuals in countries such as Canada, the UK and USA continue to want criminal offenders to receive stiffer sentences despite growing prison populations and some indication of lower crime rates (Cullen, Fisher & Applegate, 2000; Donohue, 2007; King, 2008; Raphael, 2009; Tseloni et al., 2010; Useem et al., 2003; Walmsley, 2009). Criminological research has identified cognitive and affective pathways that predict punitiveness toward crime, such as the judged wrongfulness and harmfulness of crime, and moral outrage (Carlsmith & Darley, 2008). The overall contribution of the five papers presented in this thesis is to identify the cognitive, affective and behavioural pathways that link social perception of criminals to punitiveness toward crime. Working at the intersection of social psychology and criminology, the thesis applies theoretical frameworks such as the Stereotype Content Model (Fiske, Cuddy, Glick & Xu, 2002) and Behaviour from Intergroup Affect and Stereotypes map (Cuddy, Fiske & Glick, 2007) to identify the functional relation between social perception and punitiveness. Using different methodologies and at different levels of analysis, this thesis provides strong evidence that the content of criminal stereotypes is associated with specific cognitive (e.g., perceiving crime as being more serious), affective (e.g., feeling anger and a lack of compassion) and behavioural (e.g., wanting to exclude and attack) responses. In turn, criminal stereotypes and their outcomes engender punitive intuitions, decisions and attitudes. These findings reconcile extant criminological research on punitiveness with social psychological research on the function of social stereotypes. This thesis also speaks more broadly to the association between punitiveness toward crime and basic social psychological processes related to interpersonal perception and relations. In this respect, this thesis makes a significant contribution to the study of punitiveness toward crime and has important social policy implications

An Operational Perspective to Fairness Interventions: Where and How to Intervene

As AI-based decision systems proliferate, their successful operationalization requires balancing multiple desiderata: predictive performance, disparity across groups, safeguarding sensitive group attributes (e.g., race), and engineering cost. We present a holistic framework for evaluating and contextualizing fairness interventions with respect to the above desiderata. The two key points of practical consideration are \emph{where} (pre-, in-, post-processing) and \emph{how} (in what way the sensitive group data is used) the intervention is introduced. We demonstrate our framework with a case study on predictive parity. In it, we first propose a novel method for achieving predictive parity fairness without using group data at inference time via distibutionally robust optimization. Then, we showcase the effectiveness of these methods in a benchmarking study of close to 400 variations across two major model types (XGBoost vs. Neural Net), ten datasets, and over twenty unique methodologies. Methodological insights derived from our empirical study inform the practical design of ML workflow with fairness as a central concern. We find predictive parity is difficult to achieve without using group data, and despite requiring group data during model training (but not inference), distributionally robust methods we develop provide significant Pareto improvement. Moreover, a plain XGBoost model often Pareto-dominates neural networks with fairness interventions, highlighting the importance of model inductive bias

Modeling Zero-Inflated and Overdispersed Count Data With Application to Psychiatric Inpatient Service Use

Psychiatric disorders can be characterized as behavioral or mental states that cause significant distress and impaired personal functioning. Such disorders may occur as a single episode or persistent, relapsing, and perhaps leading to suicidal behaviours. The exact causes of psychiatric disorders are hard to determine but easy access to health care services can help to reduce the severity of the states. Inpatient psychiatric hospitalization is not only an expensive mode of treatment but also may represent the quality of health care system. The aim of this study was to investigate the factors associated with repeated hospitalizations among the patients with psychiatric illness, which may help the policy makers to target the high-risk groups in a more focused manner. The count of hospitalizations for psychiatric patients may be zero during a period of time for the huge majority of patients rather than a positive count. A common strategy to handle excessive zeros is to use zero-inflated models or hurdle models. In the field of health services research of mental health, very little literature is available comparing the relative fits of zero-inflated distributions and other count distributions to empirical data. A large linked administrative database consisting of 200,537 patients with psychiatric diagnosis in the years of 2008-2012 was used in this thesis. Various counts regression models were considered for analyzing the hospitalization rate among patients with psychiatric disorders within 3, 6 and 9 months follow-up since index visit date. The covariates for this study consist of sociodemographic and clinical characteristics of the patients. According to the Akaike Information Criteria, Vuong’s test and randomized quantile residuals, the hurdle negative binomial model was the best model. Our results showed that hospitalization rate depends on the patients’ socio-demographic characteristics and also on disease types. It also showed that having previously visited a general physician served a protective role for psychiatric hospitalization during our study period. Patients who had seen an outpatient psychiatrist were more likely to have a higher number of psychiatric hospitalizations. This may indicate that psychiatrists tend to see patients with more severe illnesses, who require hospital-based care for managing their illness. Having earlier and greater access to outpatient psychiatrist and community-based mental health care may alleviate the need for hospital-based psychiatric care

Extensions and Applications of Ensemble-of-trees Methods in Machine Learning

Ensemble-of-trees algorithms have emerged to the forefront of machine learning due to their ability to generate high forecasting accuracy for a wide array of regression and classification problems. Classic ensemble methodologies such as random forests (RF) and stochastic gradient boosting (SGB) rely on algorithmic procedures to generate fits to data. In contrast, more recent ensemble techniques such as Bayesian Additive Regression Trees (BART) and Dynamic Trees (DT) focus on an underlying Bayesian probability model to generate the fits. These new probability model-based approaches show much promise versus their algorithmic counterparts, but also offer substantial room for improvement. The first part of this thesis focuses on methodological advances for ensemble-of-trees techniques with an emphasis on the more recent Bayesian approaches. In particular, we focus on extensions of BART in four distinct ways. First, we develop a more robust implementation of BART for both research and application. We then develop a principled approach to variable selection for BART as well as the ability to naturally incorporate prior information on important covariates into the algorithm. Next, we propose a method for handling missing data that relies on the recursive structure of decision trees and does not require imputation. Last, we relax the assumption of homoskedasticity in the BART model to allow for parametric modeling of heteroskedasticity. The second part of this thesis returns to the classic algorithmic approaches in the context of classification problems with asymmetric costs of forecasting errors. First we consider the performance of RF and SGB more broadly and demonstrate its superiority to logistic regression for applications in criminology with asymmetric costs. Next, we use RF to forecast unplanned hospital readmissions upon patient discharge with asymmetric costs taken into account. Finally, we explore the construction of stable decision trees for forecasts of violence during probation hearings in court systems