Bibal, Adrien

De Streel, Alexandre

Frénay, Benoît

Lognoul, Michaël

English

The requirements on explainability imposed by European laws and their implications for machine learning (ML) models are not always clear. In that perspective, our research analyzes explanation obligations imposed for private and public decision-making, and how they can be implemented by machine learning techniques

Repository of the University of Namur

RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHEAuthor(s) - Auteur(s) :Publication date - Date de publication :Permanent link - Permalien :Rights / License - Licence de droit d’auteur :Bibliothèque Universitaire Moretus PlantinInstitutional Repository - Research PortalDépôt Institutionnel - Portail de la Rechercheresearchportal.unamur.beUniversity of NamurImpact of Legal Requirements on Explainability in Machine LearningBIBAL, ADRIEN; LOGNOUL, Michael; DE STREEL, Alexandre; FRENAY, BENOITPublished in:Proceedings of the 37th International Conference on Machine LearningPublication date:2020Document VersionPeer reviewed versionLink to publicationCitation for pulished version (HARVARD):BIBAL, ADRIEN, LOGNOUL, M, DE STREEL, A & FRENAY, BENOIT 2020, Impact of Legal Requirements onExplainability in Machine L arning. in Proceedings of the 37th International Conference on Machine Learning.Proceedings of Machine Learning Research, vol. 108, MLResearch Press, Thirty-seventh InternationalConference on Machine Learning, 13/07/20.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 25. May. 2021Impact of Legal Requirements on Explainability in Machine LearningAdrien Bibal * 1 Michael Lognoul * 2 Alexandre de Streel 2 Benoı̂t Frénay 11. Legal Requirements on ExplainabilityThe requirements on explainability imposed by Europeanlaws and their implications for machine learning (ML) mod-els are not always clear. In that perspective, our research(Bibal et al., Forthcoming) analyzes explanation obligationsimposed for private and public decision-making, and howthey can be implemented by machine learning techniques.For decisions adopted by firms or individuals, we mainlyfocus on requirements imposed by general European legis-lation applicable to all the sectors of the economy. The obli-gations of the General Data Protection Regulation (GDPR)(art. 13-15 and 22) as interpreted by the European DataProtection Board (EDPB) require the processors of personaldata to provide “the rationale behind or the criteria relied onin reaching the decision,” under certain circumstances, whena fully automated decision is made (EDPB Guidelines of3 October 2017 on Automated individual decision-makingand Profiling, p. 25; see also (Edwards & Veale, 2018;Wachter et al., 2017)). Consumer protection law imposes toonline marketplaces to provide their consumers with “themain parameters determining ranking [...] and the relativeimportance of those parameters” (art. 6(a) of Directive2011/83). The Online Platforms Regulation imposes verysimilar obligations to online intermediation services andsearch engines towards their professional users (art. 5 ofRegulation 2019/1150).Sectoral rules are also analyzed. For instance, financial reg-ulators “may require the investment firm to provide [...] adescription of the nature of its algorithmic trading strate-gies, details of the trading parameters or limits to which thesystem is subject, the key compliance and risk controls thatit has in place [...]. The competent authority [...] may, atany time, request further information from an investmentfirm about its algorithmic trading and the systems used forthat trading” (art. 17(2) of Directive 2014/65 on Markets inFinancial Instruments).*Equal contribution 1PReCISE, Faculty of Computer Science,NADI, University of Namur, Belgium 2CRIDS, Faculty of Law,NADI, University of Namur, Belgium. Correspondence to: AdrienBibal <adrien.bibal@unaur.be>.Proceedings of the 37 th International Conference on MachineLearning, Vienna, Austria, PMLR 108, 2020. Copyright 2020 bythe author(s).For decisions adopted by public authorities, two strongerrequirements are studied: motivation obligations for admin-istrations and for judges (imposed by European Conventionon Human Rights). For administrative decisions, all factualand legal grounds on which the decision is based should beprovided. For judicial decisions, judges have in addition toanswer the arguments made by the parties in the litigation.The objectives of those explanation requirements aretwofold: first, allowing the recipients of a decision to un-derstand it and act accordingly; second, allowing the publicauthority, before which a decision is contested, to exercise ameaningful effective control on the legality of the decision(European Commission White Paper of 19 February 2020on Artificial Intelligence, p. 14).2. Legal Requirements and Machine LearningAs explained in the previous section, legal texts do notalways clearly identify the focus of the requirements. Inprivate decision making, we identified that the explainabilityof four levels of machine learning entities or concepts arementioned in legal texts (Bibal et al., Forthcoming): themain features used for a decision, all features used for adecision, how the features are combined for reaching adecision and the whole model (see Table 1).The first and weaker level of requirements is to provide themain features used for a decision. Note that the main param-eters mentioned in the legal texts refer to the features usedby a ML model. While the main features used are nativelyprovided by interpretable models such as linear models anddecision trees, some works go further and provide weaklyand strongly relevant features in linear models (John et al.,1994; Kohavi & John, 1997). In the context of black-boxmodels, the feature importance provided by the out-of-bagerror of random forests can pass these requirements, as wellas the feature importance provided through the perturbationof input feature values (Fisher et al., 2019).The second level of requirements is to provide all featuresinvolved in a decision. While providing all features usedis again natively proposed by interpretable models, thisrequirement can be difficult to achieve when the number offeatures used by the model is huge. Sparsity penalties suchas Lasso may be necessary to satisfy the requirement.Impact of Legal Requirements on Explainability in Machine LearningMain features• Directive 2011/83 on Consumer Rights, art. 6(a): obligation to provide the “main parameters” and their “relative importance”• Regulation 2019/1150 on promoting fairness and transparency for business users of online intermediation services, art. 5: obligation toprovide “the main parameters” and “the relative importance of those parameters”All features• Guidelines on automated individual decision-making and profiling: obligation to provide “the criteria relied on in reaching the decision”• Belgian law of 4 April 2014 on insurances, art. 46: obligation to provide “the segmentation criteria”Combination of featuresGuidelines on Automated individual decision-making and Profiling: obligation to provide “the rationale behind the decision”Whole modelDirective 2014/65 on Markets in Financial Instruments, art. 17: obligation to provide “information [...] about its algorithmic trading andthe systems used for that trading”Table 1. Table reproduced from (Bibal et al., Forthcoming) containing the legal texts used as examples in this paper.The third level of explainability requirements is to providethe combination of features that led to a particular decision.Again, interpretable models make it possible to check howthe features have been combined to lead to a decision. In thecontext of black-box models, techniques like LIME (Ribeiroet al., 2016) have been developed to get insights on howmodels behave locally, i.e. for a particular decision.Finally, the strongest requirement is to provide the wholemodel. In this case of strong requirement, only interpretablemodels can be used, as, by definition, black-box modelscannot be provided (e.g. if the model is non-parametric) orunderstood (e.g. in the case of neural networks).In addition to these four levels of explainability require-ments for private decisions, requirements for public deci-sions impose two additional constraints. For administrativedecisions, the legal motivation should also be provided withthe decision. This means that all factual and legal groundson which the decision is based must be provided. In thecase of judicial decisions, in addition to the facts of the caseand the motivation, which was already needed for adminis-trative decisions, answers to the arguments of the parties tothe litigation must also be provided. While some works tryto tackle these requirements (e.g. (Ashley & Brüninghaus,2009) explain decisions with facts only; (Zhong et al., 2018)introduce multi-task learning for dealing with legal arti-cles, as well as facts; and (Ye et al., 2018) use sequence-to-sequence learning to propose answers to the argumentsof the parties), legal requirements on the explainability ofpublic decisions remain a challenge in machine learning, be-cause ML algorithms are not designed to manipulate factualand legal grounds, as well as arguments, directly.In conclusion, we call for an interdisciplinary conversationbetween the legal and AI research communities. In particu-lar, legal scholars could benefit from better understandingthe potential and the limitations of ML models and AI schol-ars from better understanding the objectives and ambiguitiesof the law.ReferencesAshley, K. D. and Brüninghaus, S. Automatically clas-sifying case texts and predicting outcomes. ArtificialIntelligence and Law, 17(2):125–165, 2009.Bibal, A., Lognoul, M., de Streel, A., and Frénay, B. Le-gal requirements on explainability in machine learning.Artificial Intelligence and Law, Forthcoming.Edwards, L. and Veale, M. Enslaving the algorithm: From a“right to an explanation” to a “right to better decisions”?IEEE Security & Privacy, 16(3):46–54, 2018.Fisher, A., Rudin, C., and Dominici, F. All models arewrong, but many are useful: Learning a variable’s impor-tance by studying an entire class of prediction modelssimultaneously. JMLR, 20(177):1–81, 2019.John, G. H., Kohavi, R., and Pfleger, K. Irrelevant featuresand the subset selection problem. In Proceedings ofICML, pp. 121–129, 1994.Kohavi, R. and John, G. H. Wrappers for feature subsetselection. Artificial Intelligence, 97(1-2):273–324, 1997.Ribeiro, M. T., Singh, S., and Guestrin, C. “Why should Itrust you?”: Explaining the predictions of any classifier.In Proceedings of ACM SIGKDD, pp. 1135–1144, 2016.Wachter, S., Mittelstadt, B., and Floridi, L. Why a right toexplanation of automated decision-making does not existin the general data protection regulation. InternationalData Privacy Law, 7(2):76–99, 2017.Ye, H., Jiang, X., Luo, Z., and Chao, W. Interpretable chargepredictions for criminal cases: Learning to generate courtviews from fact descriptions. In Proceedings of NAACL,pp. 1854–1864, 2018.Zhong, H., Zhipeng, G., Tu, C., Xiao, C., Liu, Z., and Sun,M. Legal judgment prediction via topological learning.In Proceedings of EMNLP, pp. 3540–3549, 2018.

Impact of Legal Requirements on Explainability in Machine Learning

https://pure.unamur.be/ws/files/50501016/Abstract_LML_Workshop_Explainability_AI_Law.pdf

Impact of Legal Requirements on Explainability in Machine Learning

Abstract

Similar works

Full text

Available Versions

Repository of the University of Namur