The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning

The nascent field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last several years, three formal definitions of fairness have gained prominence: (1) anti-classification, meaning that protected attributes---like race, gender, and their proxies---are not explicitly used to make decisions; (2) classification parity, meaning that common measures of predictive performance (e.g., false positive and false negative rates) are equal across groups defined by the protected attributes; and (3) calibration, meaning that conditional on risk estimates, outcomes are independent of protected attributes. Here we show that all three of these fairness definitions suffer from significant statistical limitations. Requiring anti-classification or classification parity can, perversely, harm the very groups they were designed to protect; and calibration, though generally desirable, provides little guarantee that decisions are equitable. In contrast to these formal fairness criteria, we argue that it is often preferable to treat similarly risky people similarly, based on the most statistically accurate estimates of risk that one can produce. Such a strategy, while not universally applicable, often aligns well with policy objectives; notably, this strategy will typically violate both anti-classification and classification parity. In practice, it requires significant effort to construct suitable risk estimates. One must carefully define and measure the targets of prediction to avoid retrenching biases in the data. But, importantly, one cannot generally address these difficulties by requiring that algorithms satisfy popular mathematical formalizations of fairness. By highlighting these challenges in the foundation of fair machine learning, we hope to help researchers and practitioners productively advance the area

Free-Sorting of Colors Across Cultures: Are there Universal Grounds for Grouping?

These studies examined naming and free-sorting behavior by informants speaking a wide range of languages, from both industrialized and traditional cultures. Groups of informants, whose color vocabularies varied from 5 to 12 basic terms, were given an unconstrained color grouping task to investigate whether there are systematic differences between cultures in grouping behavior that mirror linguistic differences and, if there are not, what underlying principles might explain any universal tendencies. Despite large differences in color vocabulary, there were substantial similarities in grouping behavior across language groups, and substantial within-language variation across informants. It seems that all informants group stimuli based on some criterion of perceptual similarity, but those with large color vocabularies are more likely to group stimuli in line with their basic color terms. The data are best accounted for by a hybrid system that combines a universal principle of grouping by similarity with culture-specific category salience

A note on the Pauli problem in light of approximate joint measurements

We show that there exist informationally incomplete phase space observables such that the Cartesian margins are informationally equivalent with position and momentum. This shows that it is possible to reconstruct the position and momentum distributions of a quantum system from the statistics of a single observable, and thus a single measurement, even though the state of the system is not uniquely determined by the statistics.Comment: 12 page

Are Coping Strategies a Cop Out?

Summary Few would dispute the legitimacy of putting indigenous coping strategies firmly on the food security and famine mitigation agenda. Yet the idea of coping strategies needs to be regarded with caution, especially when used to identify famine early warning indicators. There is a tendency for ‘coping strategies’ to become shorthand for a complex web of processes at work, making for great confusion in identifying what is being talked about. With respect to famine early warning, the complexity of these processes makes for potentially huge difficulties in data collection and interpretation. In the quest to incorporate coping strategies into policy?making and planning, hard choices have to be made between operational feasibility and confronting the intricacies of ? and limitations to ? indigenous response, close to where the action is; and it is essential to be clear as to what they are. A first step is to distinguish at a conceptual level between coping and adapting. If coping strategies are to be monitored as part of early warning systems, it is the motivation for, the timing of and the effectiveness of their use which must be tracked: a formidable task. Resumé Les Stratégies D'Adaptation: Sont?Elles Tout Simplement Échappatoires? Peu de gens ne nieraient la légitimité d'inclure les stratégies autochtones adaptation aux problèmes à l'ordre du jour de tout programme de sécurité alimentaire et d'allégement de la famine. Ceci dit, il y aurait néanmoins lieu de se méfier tant soit peu de ces stratégies, surtout comme première alerte de famine. Une tendance existe par laquelle le terme stratégie d'adaptation (‘coping strategy’) est de plus en plus emprunté comme fourre?tout linguistique qui avale tout un éventail de transactions très complexes; ce qui rend l'identification exacte de ces stratégies d'autant plus compliqée. En ce qui concerne les systèmes d'alerte précoce de famine, la complexité même de ces stratégies présente de très graves difficultés au niveau du rassemblement et de l'analyse des données. Dans toute tentative d'inclure des stratégies d'adaptation dans la formulation et la mise en viguer des politiques, des choix difficiles s'imposent entre la faisabilité opérationnelle et le moyen de tenir compte des détours de la réponse autochtone et dans un même temps, de tenir compte de ses propres limites; ainsi, il est essentiel de connaitre ces détours et ces limites à fond. Un premier pas serait d'établir, au niveau conceptuel, la différence entre les stratégies d'adaptation en tant que réponses conjoncturelles face aux conditions anormales, et le processus d'adaptation permanente qui entraine des changements fondamentaux dans les systèmes de production concernés. Si l'on veut que les stratégies d'adaptation constituent un élément qui puisse être compris dans le système de'alerte précoce, il y aurait lieu de faire le suivi de la motivation, de l'incidence et de l'efficacité de leur exploitation, ce qui représenterait une tache de recherche très onéreuse. Resumen Estrategias De Solución: Una ‘Salida Fácil?’ Dentro de la agenda de soluciones para los problemas de seguridad alimentaria y mitigación del hambre, las estrategias indígenas tienan una legitimidad que muy pocos disputan. Sin embargo, esta idea de estrategias para resolver el problema del hambre debe ser abordada con precaución, especialmente cuando se usa para identificar indicadores precoces de hambre. Hay una tendencia a transformar las estrategias de solución en una versión condensada de una compleja red de procesos en acción, creando gran confusión al tratar de individualizar el asunto específico de que se trata. Con respecto a indicadores precoces, la complejidad de estos procesos ocasiona dificultades potencialmente graves en la compilación e interpretación de datos. Al tratar de incorporar estrategias dentro de la elaboración y planeamiento de programas de acción, se deben llevar a cabo elecciones muy difíciles entre la viabilidad operativa y la confrontación con la intricada ? y limitada ? respuesta local, y es esencial tener en claro qué es cada una. El primer paso es distinguir a nivel conceptual entre solución y adaptación. Para regular estas estrategias como parte del sistema de aviso precoz, se debe seguir la trayectoria de su motivación, su oportinidad y su eficacia; y ésta es una tarea formidable

In search of professionalism, bureaucracy and sustainable livelihoods for the 21st century

Summary In the 21st century, vastly more people will need to gain sustainable livelihoods in difficult environments. For this, changes are needed on the part of outsiders — professionals, officials and others — who have overlooked or underestimated the complexity and diversity of rural conditions, rural people's ability to take a long?term view, and their knowledge, creativity and competence in presenting and analysing complex information. To serve poor people better, much of the challenge now is methodological: for better learning from, with and by rural people, for enabling them to express and enhance their competence and creativity; and for the improvement and spread of new methods for changing the behaviour and attitudes of outsiders. Resumen En busca de profesionalismo, burocracia y sustento permanente para el siglo XXI En el siglo XXI, muchísimas mas personas deberán alcanzar sustento permanente en medio ambientes difíciles. Para esto, deberá haber cambios por parte de algunos sectores — profesionales, oficiales y otros — que han pasado por alto o subestimado la diversidad y complejidad de las condiciones rurales, la habilidad de los pobladores rurales para asumir una visión a largo plazo, y sus conocimientos, creatividad y competencia para presentar y analizar información intrincada. Para servir mejor a las comunidades carenciadas, la mayor parte del desafío actual es metodológica: para un mejor aprendizaje de, con y por pobladores rurales; para permitirles expresar e incrementar su competencia y creatividad; y para el mejoramiento y extensión de nuevas formas para cambiar el comportamiento y las posturas políticas de los sectores diversos. Résumé À la recherche du professionnalisme, la bureaucratie et le maintien soutenu au 21ème siècle Au 21ème siècle un nombre fortement accru d'êtres humains chercheront à exploiter des moyens soutenus d'existence dans des environnements difficiles. Ainsi, des changements doivent intervenir au niveau des tiers — professionnels, officiels et autres — qui auront soit passé outre, soit sous?estimé la complexité et la diversité des conditions rurales, ainsi que la capacité des ruraux mêmes de déterminer les politiques à long terme; leur propre créativité; et leur expertise en ce qui concerne la présentation et l'analyse des informations complexes. Si l'on veut offrir un meilleur service aux populations déshéritées, le défi sera désormais de savoir quelle méthodologie est la mieux adaptée non seulement à recevoir ce qu'ont à nous apprendre les peuples ruraux (tout en partageant cet apprentissage avec eux et en leur transferant nos propres connaissances); mais aussi, quelle méthodologie leur permettra d'exprimer et de mettre en valeur leur expertise et leur créativité; et pourra enfin améliorer et propager les nouvelles méthodes capables de modifier le comportement et les attitudes des tiers venus de l'extérieur