Effective Protection of Fundamental Rights in a pluralist worl

Fosch Villaronga, E.

Kurpicz-Briki, M.

Puttick, A.

Rigotti, C.

Leiden University Scholary Publications

Mitigating diversity biases of AI in the labor marketRigotti, C.; Puttick, A.; Fosch Villaronga, E.; Kurpicz-Briki, M.; Alvarez, J.M.; Fabris, A.; ... ;Zehlike, M.CitationRigotti, C., Puttick, A., Fosch Villaronga, E., & Kurpicz-Briki, M. (2023). Mitigatingdiversity biases of AI in the labor market. Ceur Workshop Proceedings. Retrieved fromhttps://hdl.handle.net/1887/3718485 Version: Publisher's VersionLicense: Creative Commons CC BY 4.0 licenseDownloaded from: https://hdl.handle.net/1887/3718485 Note: To cite this publication please use the final published version (if applicable).Mitigating Diversity Biases of AI in the Labor MarketCarlotta Rigotti1, Alexandre Puttick2, Eduard Fosch-Villaronga1 andMascha Kurpicz-Briki21Leiden University, Rapenburg 70, 2311 EZ Leiden, Netherlands2Berner Fachhochschule BFH, Technik und Informatik, Quellgasse 21, 2501 Biel, SwitzerlandAbstractIn recent years, artificial intelligence (AI) systems have been increasingly utilized in the labor market,with many employers relying on them in the context of human resources (HR) management. However,this increasing use has been found to have potential implications for perpetuating bias and discrimination.The BIAS project kicked off in November 2022 and is expected to develop an innovative technology(hereinafter: the Debiaser) to identify and mitigate biases in the recruitment process. For this purpose,an essential step is to gain a nuanced understanding of what constitutes AI bias and fairness in thelabor market, based on cross-disciplinary and participatory approaches. What follows is a preliminaryoverview of the design and expected implementation of the project, as well as how our project aims tocontribute to the existing literature on law, AI, bias, and fairness.Keywordsbias, fairness, discrimination, trustworthy AI, labor market, natural language processing1. Setting the scene: The increasing deployment of AIapplication in the labor marketEmployers are integrating AI applications into their HR processes to increase efficiency and cutcosts [1, 2], with the lockdown measures in response to the Covid-19 pandemic accelerating thisprocess for public health protection [3]. AI applications may be deployed in the labor market inorder to identify the best job applicant or monitor and manage the behavior and performanceof workers; they can design job descriptions, market vacancies to potential candidates, providetechnical support through chatbots, analyze large volumes of applicant data, and conductinterviews [4]. Furthermore, they can track, train, and rate workers, monitor infractions leadingto disciplinary proceedings, and predict resignation or health problems [5, 6, 7, 8].Against this backdrop, Ajunwa [9] defines the deployment of AI applications in the labormarket as ‘a black box at work,’ which lacks transparency, accountability or explanation aboutmonitoring practices. More generally, a growing body of literature voices concern aboutprivacy [10, 3, 8] and fairness [11], in the sense that AI applications are likely to perpetuatesocial marginalization and discrimination. Prejudices and flaws in past datasets and humanprogrammers can indeed easily seep into code [12, 13].EWAF’23: European Workshop on Algorithmic Fairness, June 07–09, 2023, Winterthur, SwitzerlandEnvelope-Open c.rigotti@law.leidenuniv.nl (C. Rigotti); alexandre.puttick@bfh.ch (A. Puttick);e.fosch.villaronga@law.leidenuniv.nl (E. Fosch-Villaronga); mascha.kurpicz@bfh.ch (M. Kurpicz-Briki)Orcid 0000-0001-8956-0677 (C. Rigotti); 0000-0002-2142-0309 (A. Puttick); 0000-0002-8325-5871 (E. Fosch-Villaronga);0000-0001-5539-6370 (M. Kurpicz-Briki)© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).CEURWorkshopProceedingshttp://ceur-ws.orgISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)2. Gaining new knowledge about AI bias and fairness in thelabor marketAddressing a lack of consensus about the interpretation of AI bias and fairness in the labormarket, we begin our research with a systematic literature review. In brief, we draw on thetraditional distinction of individual fairness versus group fairness [14], disparate treatmentversus disparate impact [15, 16, 12], pre-existing, technical, and emergent bias [17] versus othershapes and forms, including representation bias arising from the way we define and sample apopulation and temporal bias stemming from differences in populations and behaviors overtime [18]. In doing so, we acknowledge that the nature and significance of bias and fairness inhumans versus in AI do not overlap, to the extent that some research radically suggests theunsuitability of mathematics to capture the full meanings of these social concepts [19]. ForAbu-Elyounes [14], AI fairness is contextual and requires balancing between competing valueson a case-by-case basis, necessitating specialized consideration of the employment sector withinthe BIAS project.In validating and contextualizing the research outputs of the literature review, we also performsome fieldwork. To be precise, we have interviewed 70 HR managers and AI developers, whoare located in Estonia, Iceland, Italy, the Netherlands, Norway, Switzerland, and Turkey, toascertain their views on the use of AI applications in the labor market and its impact on fairness.Briefly, it appears that most respondents have a positive attitude towards the deploymentof AI applications for recruitment and selection purposes, while voicing concern about themanagement of staff. Although lacking a common understanding of fairness, a number ofrequirements are listed for AI-driven human resources practices to be considered fair, especiallyin terms of human oversight and transparency. Also, respondents call for the adoption ofmitigation measures to address diversity biases, with special emphasis on gender bias. BesidesAI-driven solutions similar to the Debiaser, they cite the examples of diversity quotas, DI officers,specific guidelines, and training on the matter. At the same time, we have drafted a Qualtricssurvey of workers’ attitudes towards AI, the aim of which is to understand workers’ experiencesand opinions of AI applications used in an HR management context and how they might leadto bias and discrimination. Although the survey has just been launched and will remain openuntil July, it has been reported that the majority of respondents have a fairly negative attitudetowards the use of AI applications in the labor market. In any case, for AI-driven HR practicesto be considered fair, they believe in the provision of equal opportunities despite the personalcharacteristics of the individual as well as the transparency and explicability of the technologymaking a specific decision.3. Designing the ‘Debiaser’Based on the literature review and the fieldwork, the next step of the BIAS project will be thedesign and piloting of the Debiaser. In doing so, we contribute to an existing body of researchsuggesting possible remedies for AI bias in both social and computer science [18]. The Debiaseris conceptualized as a tool for bias detection and mitigation in the context of a common use casefor text-based AI recruitment tools—the automatic pre-selection of job applicants for first-round2interviews. The tool will be built upon the principles of trustworthy AI [20] and incorporate therich knowledge base assembled by consortium partners in the framework of the BIAS project.A major contribution of our work will be the extension and adaptation of existing methods toEU regional languages and local prejudices, with a focus on specialized methods targeting fairrecruitment practices.The scientific research we conduct explores weak points in existing AI recruitment tools.The first concerns off-the-shelf word embeddings such as Word2Vec [21] and pre-trained largelanguage models (PLMs) such as BERT [22] or GPT-3 [23]. Highly capable text processingmodels require extensive resources in terms of time, computing power and data, and thus manyAI applications, including recruitment tools, are built using such word embeddings or PLMs.These machine-learning-based methods often reproduce prejudices existing in their trainingcorpora, with a notorious example being Amazon’s own (now scrapped) job candidate selectiontool, which was demonstrated to systematically discriminate against women [24]. We aimto utilize and further develop methods for detecting, measuring and mitigating bias in wordembeddings and PLMs, such as those introduced by Caliskan et al. [25], Guo and Caliskan [26]and Ahn and Oh [27]. This builds upon previous work which extends the methods of Caliskanet al. [25] to French, German, Italian and Swedish word embeddings [28, 29].Most current AI applications make use of black-box neural network models whose decision-making process is very difficult to distill into human-understandable explanations. This inhibitstrust in such systems and complicates the task of auditing algorithmic decisions. To thisend, we aim to utilize state-of-the-art methods in explainable natural language processing(XNLP) [30], aiming to provide clear explanations for all stages of bias detection and mitigationdeployed within the Debiaser. The XNLP methods we will integrate can be sorted into two basicapproaches toward improving the interpretability of algorithmic decisions: Feature-selectionmethods [31, 32, 33] aim to identify which words or phrases in the input data played animportant role in a given model’s output decision. The second category of techniques generatenatural language explanations which should explain the model’s decision in comprehensible,plain-language terms [34, 35]. In addition, our research will target the detection of text thatis susceptible to unfair discrimination and explore the potential of suggested rewrites as atechnique for avoiding such discrimination, using reinforcement learning-based techniquesin the same vein as Sharma et al. [36]. Both the suggested rewrites and natural languageexplanations generated by the Debiaser can be further improved using human feedback tofurther train the underlying models with the goal of generating output corresponding to userpreferences [37, 38].4. ConclusionThe growth of the role of AI in HR management is to continue, but its vast potential comes withvarious challenges. AI applications are far from being free from bias and discrimination, andAI developers and HR executives should be prepared to grapple with difficult questions andensure that this technology is implemented in a responsible manner. For this purpose, the BIASproject aims at identifying and mitigating diversity biases in the labor market, by combiningcross-disciplinary fieldwork and designing new technological solutions.3AcknowledgmentsThis work is part of the Europe Horizon project BIAS funded by the European Commission, andhas received funding from the Swiss State Secretariat for Education, Research and Innovation(SERI).References[1] N. Guenole, S. Feinzig, The business case for ai in hr: With insights and tips on gettingstarted, IBM Smarter Workforce Institute (2018).[2] N. T. Tippins, F. L. Oswald, S. M. McPhail, Scientific, legal, and ethical concerns aboutai-based personnel selection tools: a call to action, Personnel Assessment and Decisions 7(2021) 1.[3] I. Ebert, I. Wildhaber, J. Adams-Prassl, Big data in the workplace: Privacy due diligenceas a human rights-based approach to employee privacy protection, Big Data & Society 8(2021) 20539517211013051.[4] M. F. Gonzalez, W. Liu, L. Shirase, D. L. Tomczak, C. E. Lobbe, R. Justenhoven, N. R.Martin, Allying with ai? reactions toward human-based, ai/ml-based, and augmentedhiring processes, Computers in Human Behavior 130 (2022) 107179.[5] M. V. V. Yawalkar, a study of artificial intelligence and its role in human resource man-agement, International Journal of Research and Analytical Reviews (IJRAR) 6 (2019)20–24.[6] S. M. C. Loureiro, J. Guerreiro, I. Tussyadiah, Artificial intelligence in business: State ofthe art and future research agenda, Journal of business research 129 (2021) 911–926.[7] B. Eubanks, Artificial intelligence for HR: Use AI to support and develop a successfulworkforce, Kogan Page Publishers, 2022.[8] A. Aloisi, Regulating algorithmic management at work in the european union: Dataprotection, non-discrimination and collective rights, International Journal of ComparativeLabour Law and Industrial Relations, Forthcoming (2022).[9] I. Ajunwa, The “black box” at work, Big Data & Society 7 (2020) 2053951720966181.[10] D. P. Bhave, L. H. Teo, R. S. Dalal, Privacy at work: A review and a research agenda for acontested terrain, Journal of Management 46 (2020) 127–164.[11] A. L. Hunkenschroer, C. Luetge, Ethics of ai-enabled recruiting and selection: A reviewand research agenda, Journal of Business Ethics 178 (2022) 977–1007.[12] J. Kleinberg, J. Ludwig, S. Mullainathan, C. R. Sunstein, Discrimination in the age ofalgorithms, Journal of Legal Analysis 10 (2018) 113–174.[13] M. Raub, Bots, bias and big data: artificial intelligence, algorithmic bias and disparateimpact liability in hiring practices, Ark. L. Rev. 71 (2018) 529.[14] D. Abu-Elyounes, Contextual fairness: A legal and policy analysis of algorithmic fairness,U. Ill. JL Tech. & Pol’y (2020) 1.[15] S. Barocas, M. Hardt, A. Narayanan, Fairness in machine learning, Nips tutorial 1 (2017)2017.4[16] M. MacCarthy, Standards of fairness for disparate impact assessment of big data algorithms,Cumb. L. Rev. 48 (2017) 67.[17] B. Friedman, H. Nissenbaum, Bias in computer systems, ACM Transactions on InformationSystems (TOIS) 14 (1996) 330–347.[18] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, A. Galstyan, A survey on bias and fairnessin machine learning, ACM Computing Surveys (CSUR) 54 (2021) 1–35.[19] A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, J. Vertesi, Fairness andabstraction in sociotechnical systems, in: Proceedings of the conference on fairness,accountability, and transparency, 2019, pp. 59–68.[20] A. Hleg, Ethics guidelines for trustworthy ai, B-1049 Brussels (2019).[21] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed representations ofwords and phrases and their compositionality, Advances in neural information processingsystems 26 (2013).[22] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectionaltransformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).[23] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan,P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances inneural information processing systems 33 (2020) 1877–1901.[24] J. Dastin, Amazon scraps secret ai recruiting tool that showed bias against women, in:Ethics of data and analytics, Auerbach Publications, 2018, pp. 296–299.[25] A. Caliskan, J. J. Bryson, A. Narayanan, Semantics derived automatically from languagecorpora contain human-like biases, Science 356 (2017) 183–186.[26] W. Guo, A. Caliskan, Detecting emergent intersectional biases: Contextualized wordembeddings contain a distribution of human-like biases, in: Proceedings of the 2021AAAI/ACM Conference on AI, Ethics, and Society, 2021, pp. 122–133.[27] J. Ahn, A. Oh, Mitigating language-dependent ethnic bias in bert, arXiv preprintarXiv:2109.05704 (2021).[28] M. Kurpicz-Briki, Cultural differences in bias? origin and gender bias in pre-trainedgerman and french word embeddings (2020).[29] M. Kurpicz-Briki, T. Leoni, A world full of stereotypes? further investigation on originand gender bias in multi-lingual word embeddings, Frontiers in big Data 4 (2021) 625290.[30] M. Danilevsky, K. Qian, R. Aharonov, Y. Katsis, B. Kawas, P. Sen, A survey of the state ofexplainable ai for natural language processing, arXiv preprint arXiv:2010.00711 (2020).[31] M. T. Ribeiro, S. Singh, C. Guestrin, ” why should i trust you?” explaining the predictionsof any classifier, in: Proceedings of the 22nd ACM SIGKDD international conference onknowledge discovery and data mining, 2016, pp. 1135–1144.[32] S. Rathi, Generating counterfactual and contrastive explanations using shap, arXiv preprintarXiv:1906.09293 (2019).[33] J. Li, X. Chen, E. Hovy, D. Jurafsky, Visualizing and understanding neural models in nlp,arXiv preprint arXiv:1506.01066 (2015).[34] O.-M. Camburu, T. Rocktäschel, T. Lukasiewicz, P. Blunsom, e-snli: Natural languageinference with natural language explanations, Advances in Neural Information ProcessingSystems 31 (2018).[35] S. Narang, C. Raffel, K. Lee, A. Roberts, N. Fiedel, K. Malkan, Wt5?! training text-to-text5models to explain their predictions, arXiv preprint arXiv:2004.14546 (2020).[36] A. Sharma, I. W. Lin, A. S. Miner, D. C. Atkins, T. Althoff, Towards facilitating empathicconversations in online mental health support: A reinforcement learning approach, in:Proceedings of the Web Conference 2021, 2021, pp. 194–205.[37] P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, D. Amodei, Deep reinforcementlearning from human preferences, Advances in neural information processing systems 30(2017).[38] D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano,G. Irving, Fine-tuning language models from human preferences, arXiv preprintarXiv:1909.08593 (2019).6

Mitigating diversity biases of AI in the labor market

https://scholarlypublications.universiteitleiden.nl/access/item%3A3718486/download

Mitigating diversity biases of AI in the labor market

Abstract

Similar works

Full text

Available Versions

Leiden University Scholary Publications