Language (Technology) is Power: A Critical Survey of "Bias" in NLP

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities

Knowledge in the dark: scientific challenges and ways forward

A key dimension of our current era is Big Data, the rapid rise in produced data and information; a key frustration is that we are nonetheless living in an age of ignorance, as the real knowledge and understanding of people does not seem to be substantially increasing. This development has critical consequences, for example it limits the ability to find and apply effective solutions to pressing environmental and socioeconomic challenges. Here, we propose the concept of “knowledge in the dark”—or short: dark knowledge—and outline how it can help clarify key reasons for this development: (i) production of biased, erroneous, or fabricated data and information; (ii) inaccessibility and (iii) incomprehensibility of data and information; and (iv) loss of previous knowledge. Even in the academic realm, where financial interests are less pronounced than in the private sector, several factors lead to dark knowledge, that is they inhibit a more substantial increase in knowledge and understanding. We highlight four of these factors—loss of academic freedom, research biases, lack of reproducibility, and the Scientific tower of Babel—and offer ways to tackle them, for example establishing an international court of arbitration for research and developing advanced tools for research synthesis

Predictive validity of the CriSTAL tool for short-term mortality in older people presenting at Emergency Departments: a prospective study

© 2018, The Author(s). Abstract: To determine the validity of the Australian clinical prediction tool Criteria for Screening and Triaging to Appropriate aLternative care (CRISTAL) based on objective clinical criteria to accurately identify risk of death within 3 months of admission among older patients. Methods: Prospective study of ≥ 65 year-olds presenting at emergency departments in five Australian (Aus) and four Danish (DK) hospitals. Logistic regression analysis was used to model factors for death prediction; Sensitivity, specificity, area under the ROC curve and calibration with bootstrapping techniques were used to describe predictive accuracy. Results: 2493 patients, with median age 78–80 years (DK–Aus). The deceased had significantly higher mean CriSTAL with Australian mean of 8.1 (95% CI 7.7–8.6 vs. 5.8 95% CI 5.6–5.9) and Danish mean 7.1 (95% CI 6.6–7.5 vs. 5.5 95% CI 5.4–5.6). The model with Fried Frailty score was optimal for the Australian cohort but prediction with the Clinical Frailty Scale (CFS) was also good (AUROC 0.825 and 0.81, respectively). Values for the Danish cohort were AUROC 0.764 with Fried and 0.794 using CFS. The most significant independent predictors of short-term death in both cohorts were advanced malignancy, frailty, male gender and advanced age. CriSTAL’s accuracy was only modest for in-hospital death prediction in either setting. Conclusions: The modified CriSTAL tool (with CFS instead of Fried’s frailty instrument) has good discriminant power to improve prognostic certainty of short-term mortality for ED physicians in both health systems. This shows promise in enhancing clinician’s confidence in initiating earlier end-of-life discussions

Mitigating Gender Bias in Machine Learning Data Sets

Artificial Intelligence has the capacity to amplify and perpetuate societal biases and presents profound ethical implications for society. Gender bias has been identified in the context of employment advertising and recruitment tools, due to their reliance on underlying language processing and recommendation algorithms. Attempts to address such issues have involved testing learned associations, integrating concepts of fairness to machine learning and performing more rigorous analysis of training data. Mitigating bias when algorithms are trained on textual data is particularly challenging given the complex way gender ideology is embedded in language. This paper proposes a framework for the identification of gender bias in training data for machine learning.The work draws upon gender theory and sociolinguistics to systematically indicate levels of bias in textual training data and associated neural word embedding models, thus highlighting pathways for both removing bias from training data and critically assessing its impact.Comment: 10 pages, 5 figures, 5 Tables, Presented as Bias2020 workshop (as part of the ECIR Conference) - http://bias.disim.univaq.i

The Persistence of Ethnicity in African American Popular Music: A Theology of Rap Music

The racial oppression of black people in many ways has fueled and shaped black musical forms in America. One example is the blues which originated in the rural South among poor, nonliterate, agrarian African Americans.[1] In the North the music became more formalized, and singers such as Gertrude Ma Rainey, Bessie Smith, Mamie Smith, Ida Cox, and Sarah Martin became known as the queens of the classic blues. Another musical genre is jazz, which was largely based on the twelve-bar blues harmonic structure and phrasing. It was more polished than the earlier New Orleans jazz at the turn of the century, and its major influences came from New York City, Chicago, and Kansas City. Finally, on the religious front, gospel music was in its early stages of development around the time early blues was evolving. Influenced by blues and jazz, gospel was revolutionary (and controversial) in its combination of drums and fast, rocking rhythms

A risk prediction model for the assessment and triage of women with hypertensive disorders of pregnancy in low-resourced settings: the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) multi-country prospective cohort study.

BACKGROUND: Pre-eclampsia/eclampsia are leading causes of maternal mortality and morbidity, particularly in low- and middle- income countries (LMICs). We developed the miniPIERS risk prediction model to provide a simple, evidence-based tool to identify pregnant women in LMICs at increased risk of death or major hypertensive-related complications. METHODS AND FINDINGS: From 1 July 2008 to 31 March 2012, in five LMICs, data were collected prospectively on 2,081 women with any hypertensive disorder of pregnancy admitted to a participating centre. Candidate predictors collected within 24 hours of admission were entered into a step-wise backward elimination logistic regression model to predict a composite adverse maternal outcome within 48 hours of admission. Model internal validation was accomplished by bootstrapping and external validation was completed using data from 1,300 women in the Pre-eclampsia Integrated Estimate of RiSk (fullPIERS) dataset. Predictive performance was assessed for calibration, discrimination, and stratification capacity. The final miniPIERS model included: parity (nulliparous versus multiparous); gestational age on admission; headache/visual disturbances; chest pain/dyspnoea; vaginal bleeding with abdominal pain; systolic blood pressure; and dipstick proteinuria. The miniPIERS model was well-calibrated and had an area under the receiver operating characteristic curve (AUC ROC) of 0.768 (95% CI 0.735-0.801) with an average optimism of 0.037. External validation AUC ROC was 0.713 (95% CI 0.658-0.768). A predicted probability ≥25% to define a positive test classified women with 85.5% accuracy. Limitations of this study include the composite outcome and the broad inclusion criteria of any hypertensive disorder of pregnancy. This broad approach was used to optimize model generalizability. CONCLUSIONS: The miniPIERS model shows reasonable ability to identify women at increased risk of adverse maternal outcomes associated with the hypertensive disorders of pregnancy. It could be used in LMICs to identify women who would benefit most from interventions such as magnesium sulphate, antihypertensives, or transportation to a higher level of care

Towards validation of a new computerised test of goal neglect: preliminary evidence from clinical and neuroimaging pilot studies

Objective: Goal neglect is a significant problem following brain injury, and is a target for rehabilitation. It is not yet known how neural activation might change to reflect rehabilitation gains. We developed a computerised multiple elements test (CMET), suitable for use in neuroimaging paradigms. Design: Pilot correlational study and event-related fMRI study. Methods: In Study 1, 18 adults with acquired brain injury were assessed using the CMET, other tests of goal neglect (Hotel Test; Modified Six Elements Test) and tests of reasoning. In Study 2, 12 healthy adults underwent fMRI, during which the CMET was administered under two conditions: self-generated switching and experimenter-prompted switching. Results: Among the clinical sample, CMET performance was positively correlated with both the Hotel Test (r = 0.675, p = 0.003) and the Modified Six Elements Test (r = 0.568, p = 0.014), but not with other clinical or demographic measures. In the healthy sample, fMRI demonstrated significant activation in rostro-lateral prefrontal cortex in the self-generated condition compared with the prompted condition (peak 40, 44, 4; ZE = 4.25, p(FWEcorr) = 0.026). Conclusions: These pilot studies provide preliminary evidence towards the validation of the CMET as a measure of goal neglect. Future studies will aim to further establish its psychometric properties, and determine optimum pre- and post-rehabilitation fMRI paradigms