Nemitz, Paul

English

<intR>²Dok

Artificial Intelligence Must Be UsedAccording to the Law, or Not at AllPaul Nemitz 2022-03-28T11:36:19Democracy requires to strengthen the Rule of Law wherever public or private actorsuse algorithmic systems. The law must set out the requirements on AI necessaryin a democratic society and organize appropriate accountability and oversight. Tothis end, the European Commission made several legislative proposals. In additionto the discussion on how to use algorithmic systems lawfully, the question when itis beneficial to use them deserves more attention.Nobody should hide their responsibilities behindautomationIn August 2020, when hundreds of students grouped together and shouted “f**k thealgorithm” in front of the UK’s Department for Education, they did not rage againstthe machine. Their rage was directed against the government that decided to use atool they perceived as unjust. Even though media headlines and articles sometimesclaim that “algorithms already rule our lives”, the students were not confused aboutwho was responsible. Even if certain decision-making processes are automated,a “rule of the algorithm” shall and does not exist. There must always be someone,either a legal or natural person, who uses the algorithm and can be held responsible.In some cases, this use can be a challenge for the respect and enforcement ofapplicable legislation, where a system is used without adequate safeguards andquality controls to automate or support decision-making processes or for activitiessuch as surveillance, this may violate the rights of individuals. Such violations canoccur at great scale, depending on how broadly a system is used, and they canbe difficult to prevent or detect when the system is not sufficiently transparent,or people remain unaware of its use. For example, the automated inference ofinformation about people can affect their privacy and data protection rights. Anotherexample is that bias in algorithms or training data of AI systems can lead to unjustand discriminatory outcomes.The use of automated systems can also affect many other rights laid down in theEuropean Charter of Fundamental rights, such as those to human dignity, goodadministration, consumer protection, social security and assistance, freedom ofexpression, freedom of assembly, education, asylum, collective bargaining andaction, fair and just working conditions, access to preventive care, or cultural andlinguistic diversity. If those systems are used in the context of law enforcement or thejudiciary, they can also affect the presumption of innocence and the right to fair trialand defense.- 1 -The inaccessibility or non-existence of relevant information on automated systemsimpedes effective enforcement of fundamental rights obligations, secondary lawand access to legal remedies. In addition, we sometimes see a tendency to designfor purposes of deception and for evading responsibility. For example, certain carmakers designed an algorithm that recognized when a car was being tested forexhaust fumes, to produce false test results which would not correspond to the realemissions in normal traffic. And in the Right to be Forgotten/Google Spain Casebefore the European Court of Justice, Google tried to dissociate its responsibilityfrom the performance of its search algorithm, with the argument that it was fullyautomated, and that Google would not be responsible for the search results as acompany or as a controller under the General Data Protection Regulation (GDPR).Thankfully, the European Court of Justice did not accept this early effort to create anew “irresponsibility defense”.A coordinated European approach to ensure thatpeople rule the algorithms1)This section is anadapted reproduction of passages from the 2021annual report on the application of the Charter offundamental rightsTo address the challenges identified in the previous section, in April 2021, theEuropean Commission presented a proposal for a Regulation on AI (AIA). Keyobjectives of the proposed AIA are the protection of fundamental rights and safety.The proposal aims to ensure that high-risk AI systems are designed and usedin compliance with fundamental rights and that competent national authoritiesand courts can more effectively investigate and address possible breaches offundamental rights obligations.The proposal follows a risk-based approach. Certain AI systems are prohibitedoutright, such as the use of remote biometric identification systems in publiclyaccessible spaces for law enforcement purposes, unless clearly defined exceptionsand safeguards apply (Art. 5.1.d and 5.2).High-risk AI systems will need to comply with a set of requirements (Art. 8-15).Those requirements ensure appropriate documentation and testing of high-risk AIsystems, as well as adequate data quality, traceability, human oversight, robustness,accuracy and cybersecurity. They will apply where AI systems are used in criticalareas, such as biometric identification, education, employment, essential public andprivate services, such as credits, or public assistance benefits, law enforcement,migration and border control, and the judiciary (Annex III).The proposal ensures that the users of AI systems, such as companies interactingwith clients, or public authorities taking decisions, are provided with adequateinformation from the developers of the systems to ensure suitable use of their- 2 -applications and to enable them to fulfil their obligations under fundamental rightslaw (Art. 13).Should infringements of fundamental rights occur through the use of AI systems,effective redress for affected persons will be facilitated by means of transparencyand traceability of AI systems, coupled with strong ex post controls by competentauthorities. Supervisory authorities in charge of enforcing fundamental rights, suchas data protection authorities, equality bodies or consumer bodies, will have accessto all documentation on high-risk AI systems that fall within their mandate and theywill be able to cooperate with market surveillance authorities to test the respective AIsystems where needed (Art. 64).For specific AI systems, transparency obligations (Art. 52) towards affected peoplewill minimize the risk of manipulation, in particular, in the case of chat bots (computerprograms that can answer questions in an online chat) or ‘deep fakes’ (artificiallygenerated or manipulated image, audio or video content that resembles existingpeople, objects, places or other entities or events and which falsely appear to beauthentic or truthful).The AIA proposal is currently under negotiation with the co-legislators. It willwork jointly with other existing and proposed legislation laying down substantiverules for the use of AI systems in clearly targeted contexts, and it will work jointlywith other legislation that is applicable to automated systems without havingspecific rules on the technology or its use. It is important to look at the AIA proposaltogether with these other elements, because the different initiatives and laws aredesigned to complement and strengthen each other. When the Commission receivescriticism about lacking provisions in a particular proposal, the explanation for thislack is sometimes that the matter is addressed elsewhere. Given the increasingimportance of automation in different areas of daily life, the corpus of relevant lawsand proposals is growing.For example, when automated systems are used to process personal data, theGDPR is fully applicable. Many of the rights and obligations set out in the GDPRwill thus have to be respected in the design and use of AI systems, such as theinformation rights, including notably in case of automated processing that affectsrights of individuals, as set out in Articles 13-15 and 22 of the GDPR.When automated systems are used on platforms like social networks, this will haveto be done in compliance with the proposal for a Digital Services Act, adopted bythe Commission in December 2020, which is currently under discussion by the co-legislators. It frames the responsibilities of online intermediaries. Without prejudiceto sector-specific EU rules such as those on copyright or terrorist content online,it provides a single horizontal set of rules in the EU for a balanced governance ofonline content moderation.This would apply jointly with the proposed Regulation on transparency and targetingof political advertising, adopted by the European Commission in November 2021,as part of measures aimed at protecting election integrity and open democraticdebate. These proposed rules would require any political advertisement to be clearly- 3 -labelled as such and include information such as who paid for it and how much.Political targeting and amplification techniques would need to be explained publiclyin unprecedented detail and would be banned when using sensitive personal datawithout the explicit consent of the individual.In addition, in June 2021, the European Commission adopted a new proposal fora Directive on consumer credits repealing and replacing the current ConsumerCredit Directive. It proposes rules in relation to granting credits to consumersaccording to which Member States will have to ensure documentation of proceduresand information used in creditworthiness assessments (Art. 18.3). Furthermore,the assessments will have to be based on relevant and accurate information onfinancial and economic circumstances (e.g. income and expenses) and should notbe based on data such as social media data (Art 18.2 and recital 47). Consumerswill also have the right to an explanation (Art 18.6) on how a decision on theircreditworthiness was reached, to express their point of view and to obtain humanintervention, mirroring the principles of the GDPR concerning automated decision-making. The proposal is currently under negotiation with the co-legislators.In December 2021, the European Commission has proposed a Directive toimprove working conditions for platform workers at the EU level by ensuring correctdetermination of their employment status, by promoting transparency, fairnessand accountability in algorithmic management in platform work and by improvingtransparency in platform work. This proposal is also under negotiation with the co-legislators.There is much more work aheadTo enforce the abovementioned laws and other rules, states, rights holders and civilsociety will face an increasing need for technological capacities. Those who needto comply with these rules will have similar demands. We are not at the advent ofany “rule of the algorithm”. However, in the face of increasingly complex automatedsystems used in ways that are relevant for the functioning of our societies, it willtake great efforts to rule over the algorithms used already today. The law will haveto be further developed, alongside new technologies to support its application andenforcement.The law is a noble expression of democracy and we must ensure its enforcement. Atthe same time, democracy and the principle that the way things are done in a societymust be open for change also has implications for the use of AI and other forms ofautomation.In the light of these challenges, it seems useful to examine in each particularsituation whether the use of a specific automated system is actually an improvement.As Julia Powles observed, when it comes to artificial intelligence, there is oftena lot of discussion on how to use it, but very little on whether using it would beappropriate: “The endgame is always to “fix” A.I. systems, never to use a differentsystem or no system at all.”- 4 -There is so much rhetoric about the declared benefits of AI tools, notablyabout efficiency, it sometimes seems as if there was a common belief that anyAI system would always amount to an efficient solution, notwithstanding needsfor adaptation and maintenance, the bureaucracy to ensure an appropriate datamanagement, or challenges to ensure a lawful and reliable use. This enthusiasmdeserves more scrutiny. Not every task is best fulfilled by means of automation.Especially in contexts that require genuine understanding of the objectives and thesituation at hand, automation might be less efficient and reliable.While public entities usually have to ensure good administration to the extentpossible, private parties are free to use flawed tools as long as they do not breakthe law. In hiring, for instance, a number of automated tools are marketed withpromises about the information these tools supposedly generate about applicantswhen analyzing data about them, and about the efficiency gains this would bring forpotential employers. Often enough, there is little evidence to back these claims. Atthe same time, there is a growing realization in academia and among companies thatemployers are missing out on good candidates when they use certain software toscan CVs. If a tool is already implemented and does not work as intended, it can bechallenging to fix it, especially when it is provided by a third party.Another way to avoid the “rule of the algorithm” is therefore not to use one whereit does not work properly or even causes a loss of control over the way tasks arecarried out. Thoroughly examining its suitability before deploying an automatedsystem is good advice also beyond legal compliance.Automation in the public sector in countries such as Germany usually requiresa specific legal basis. So far, these only exist where the law is very “numerical”,thus for example in the area of taxation and social support, when it comes tothe calculation of obligations or benefits. However, the law excludes the use ofautomated systems for administrative acts wherever these require the use ofdiscretion. Only a human can exercise discretion. This principle of binding theintroduction of automation to the need for an explicit legal basis is a key safeguardof democracy, through the rule of law, against technological automation that couldotherwise risk undermining democratic accountability.It will be key for democracies to assess any use of automated systems in theexercise of public authority, as to the affordances of the system to accommodatedemocratic political change, at which costs and with which speed. Democracy inthe smart city, for example, cannot be locked into technological systems that aretoo expensive to adapt once a new political majority takes over. Before deployingAI systems in any sector that is subject to political choices, from education viahealth through to mobility or law enforcement, we must ask the question whetherthe system will be as efficient and open to incorporating and operationalizing ademocratic change of direction in these policies as humans are. We must also askthe question whether the automated system affords criticism, interrogation, and thedrive for change born out of frustration with the present, which are key to innovativeand progressive societies. So far, it is judges who dissent, teachers and studentswho protest, or doctors who try new methods outside the traditional path, to givesome examples, which are at the core of creativity, innovation and progress. So- 5 -far, we have not seen such capacities in any technological system. We must avoidtechnological conservatism that results in a lock-in situation when moving tasks fromhumans who have those capabilities to machines, which do not. The way must beopen for change, also in the digital age.The authors express their personal opinion and not necessarily that of the EuropeanCommission.References• This section is an adapted reproduction of passages from the 2021 annualreport on the application of the Charter of fundamental rights- 6 -

Artificial Intelligence Must Be Used According to the Law, or Not at All

https://intr2dok.vifa-recht.de/servlets/MCRFileNodeServlet/mir_derivate_00012121/Artificial_Intelligence_Must_Be_Used_According_to_the_Law_or_Not_at_All.pdf

Artificial Intelligence Must Be Used According to the Law, or Not at All

Abstract

Similar works

Full text

Available Versions

<intR>²Dok