Like trainer, like bot? Inheritance of bias in algorithmic content moderation

The internet has become a central medium through which `networked publics' express their opinions and engage in debate. Offensive comments and personal attacks can inhibit participation in these spaces. Automated content moderation aims to overcome this problem using machine learning classifiers trained on large corpora of texts manually annotated for offence. While such systems could help encourage more civil debate, they must navigate inherently normatively contestable boundaries, and are subject to the idiosyncratic norms of the human raters who provide the training data. An important objective for platforms implementing such measures might be to ensure that they are not unduly biased towards or against particular norms of offence. This paper provides some exploratory methods by which the normative biases of algorithmic content moderation systems can be measured, by way of a case study using an existing dataset of comments labelled for offence. We train classifiers on comments labelled by different demographic subsets (men and women) to understand how differences in conceptions of offence between these groups might affect the performance of the resulting models on various test sets. We conclude by discussing some of the ethical choices facing the implementers of algorithmic moderation systems, given various desired levels of diversity of viewpoints amongst discussion participants.Comment: 12 pages, 3 figures, 9th International Conference on Social Informatics (SocInfo 2017), Oxford, UK, 13--15 September 2017 (forthcoming in Springer Lecture Notes in Computer Science

Epistemic Injustice in the Age of AI

Artificial Intelligence (AI) is revolutionising our practices of distributing and producing knowledge. Though promising, these technologies also harbour the potential for corruption - a rising problem in this domain is that of injustice committed against women in the epistemic sphere. In our social framework, being regarded as a credible knower has become synonymous with the potential for self-actualisation: the realisation of one’s potential. As such, the gender bias perpetrated by some AI systems is harming women in this domain. Additionally, biased software is barring them from accessing hermeneutical resources relevant to the understanding of their lived experience. Though still in its infancy, the problem should be urgently addressed by conceptualising ways in which a fairer AI could be engineered. Egalitarian ideas, specifically focused on equality of opportunity, seem to be promising avenues for future research and thought

Inequalities and content moderation

As the harms of hate speech, mis/disinformation and incitement to violence on social media have become increasingly apparent, calls for regulation have accelerated. Most of these debates have centred around the needs and concerns of large markets such as the EU and the United States, or the aggressive approach countries such as Russia and China adopt to regulate online content. Our focus in this article is with the rest, the smaller markets at the periphery of the advertising industry, and the deep inequalities that current approaches to content moderation perpetuate. We outline the depth of the unequal practice of moderation, particularly across Africa, and explore the underlying political and economic factors driving this gap. While recognizing content moderation has many limitations, we conclude by underlining potential approaches to increase oversight in content moderation.info:eu-repo/semantics/publishedVersio

Addressing contingency in algorithmic (mis)information classification: Toward a responsible machine learning agenda

Machine learning (ML) enabled classification models are becoming increasingly popular for tackling the sheer volume and speed of online misinformation and other content that could be identified as harmful. In building these models, data scientists need to take a stance on the legitimacy, authoritativeness and objectivity of the sources of ``truth" used for model training and testing. This has political, ethical and epistemic implications which are rarely addressed in technical papers. Despite (and due to) their reported high accuracy and performance, ML-driven moderation systems have the potential to shape online public debate and create downstream negative impacts such as undue censorship and the reinforcing of false beliefs. Using collaborative ethnography and theoretical insights from social studies of science and expertise, we offer a critical analysis of the process of building ML models for (mis)information classification: we identify a series of algorithmic contingencies--key moments during model development that could lead to different future outcomes, uncertainty and harmful effects as these tools are deployed by social media platforms. We conclude by offering a tentative path toward reflexive and responsible development of ML tools for moderating misinformation and other harmful content online.Comment: Andr\'es Dom\'inguez Hern\'andez, Richard Owen, Dan Saattrup Nielsen and Ryan McConville. 2023. Addressing contingency in algorithmic (mis)information classification: Toward a responsible machine learning agenda. Accepted in 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23), June 12-15, 2023, Chicago, United States of America. ACM, New York, NY, USA, 16 page

Machines do not decide hate speech: Machine learning, power, and the intersectional approach

The advent of social media has increased digital content - and, with it, hate speech. Advancements in machine learning help detect online hate speech at scale, but scale is only one part of the problem related to moderating it. Machines do not decide what comprises hate speech, which is part of a societal norm. Power relations establish such norms and, thus, determine who can say what comprises hate speech. Without considering this data-generation process, a fair automated hate speech detection system cannot be built. This chapter first examines the relationship between power, hate speech, and machine learning. Then, it examines how the intersectional lens - focusing on power dynamics between and within social groups - helps identify bias in the data sets used to build automated hate speech detection systems

The Need for Sensemaking in Networked Privacy and Algorithmic Responsibility

This paper proposes that two significant and emerging problems facing our connected, data-driven society may be more effectively solved by being framed as sensemaking challenges. The first is in empowering individuals to take control of their privacy, in device-rich information environments where personal information is fed transparently to complex networks of information brokers. Although sensemaking is often framed as an analytical activity undertaken by experts, due to the fact that non-specialist end-users are now being forced to make expert-like decisions in complex information environments, we argue that it is both appropriate and important to consider sensemaking challenges in this context. The second is in supporting human-in-the-loop algorithmic decision-making, in which important decisions bringing direct consequences for individuals, or indirect consequences for groups, are made with the support of data-driven algorithmic systems. In both privacy and algorithmic decision-making, framing the problems as sensemaking challenges acknowledges complex and illdefined problem structures, and affords the opportunity to view these activities as both building up relevant expertise schemas over time, and being driven potentially by recognition-primed decision making

Algorithmic governance in public sector: is digitization a key to effective management

While algorithmic governance in the public sector can lead to increased efficiency and cost-effectiveness, the implementation of those digital innovations can also result in multiple forms of harm: data bias can lead to reinforcement of inequality, discrimination, and criminalization of already marginalized populations; lack of accountability and transparency in decision-making can lead to injustices; societal trust and the legitimacy of public sector institutions may suffer; privacy and fundamental human rights may be threatened, ethical standards challenged. Digital transformation, leading to algorithmic governance, may be challenged in times of crisis, such as the recent pandemic outbreak, as new technologies in public sector institutions and forms of data-driven surveillance and intrusive monitoring are introduced in the name of public security and social need.  This research focuses in affirming the assumption that the effective management in the public sector, first of all, is determined by the ability of this sector to transform the perception of the services delivered; secondly, it requires strategic actions to enable the systemic and coherent digital transformation of the public sector; and lastly, the new strategies of human resources management in the public sector should be considered. The focus is concentrated on understanding how the implementation of digital tools to the public sector and public services correlate with algorithmic governance concept and what impact digitization has on the effectiveness of management in the public sector