Guizzardi, Renata

Horkoff, Jennifer

Perini, Anna

Susi, Angelo

English

University of Twente Research Information

Preface: The 4th International Workshop onRequirements Engineering for Artificial Intelligence(RE4AI’23)Renata Guizzardi1, Jennifer Horkoff2, Anna Perini3 and Angelo Susi31University of Twente, The Netherlands2Chalmers | University of Gothenburg, Sweden3Fondazione Bruno Kessler, Trento, Italy1. PrefaceIt is our pleasure to welcome you to the fourth edition of the International Workshop onRequirements Engineering for Artificial Intelligence (RE4AI’23), held in conjunction with the29th Working Conference on Requirements Engineering (REFSQ 2023), in the city of Barcelona,Spain.RE4AI aims to provide a forum for discussing how Requirements Engineering methods,techniques and tools may be used to support the development of Artificial Intelligence systemsthat are lawful, ethical and robust.Workshop Format: Based on feedback from the previous edition of this workshop, we decidedto change the format of the workshop this year, proposing sessions with the presentation oftalks by invited speakers. Many of these speakers are the authors of former papers in RE4AI,i.e., researchers who have been busy with the development of work in this area for at least 4years. Others have been invited due to the recognition that they too develop relevant researchin the area, although they haven’t yet had the opportunity to publish at this forum.Specifically we invited speakers to elaborate on their experience with RE4AI, including aconsideration of questions like: What is the role of RE in engineering AI-based systems?1 AreRE foundational concepts and theory still useful?2 As RE researchers, what can we learn from AIIn: A. Ferrari, B. Penzenstadler, I. Hadar, S. Oyedeji, S. Abualhaija, A. Vogelsang, G. Deshpande, A. Rachmann, J.Gulden, A. Wohlgemuth, A. Hess, S. Fricker, R. Guizzardi, J. Horkoff, A. Perini, A. Susi, O. Karras, A. Moreira, F. Dalpiaz,P. Spoletini, D. Amyot. Joint Proceedings of REFSQ-2023 Workshops, Doctoral Symposium, Posters & Tools Track, andJournal Early Feedback Track. Co-located with REFSQ 2023. Barcelona, Catalunya, Spain, April 17, 2023.Envelope-Open r.guizzardi@utwente.nl (R. Guizzardi); jennifer.horkoff@gu.se (J. Horkoff); perini@fbk.eu (A. Perini);susi@fbk.eu (A. Susi)Orcid 0000-0002-2019-5277 (J. Horkoff); 0000-0001-8818-6476 (A. Perini); 0000-0002-5026-7462 (A. Susi)© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).CEURWorkshopProceedingshttp://ceur-ws.orgISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)1See ideas proposed in IEEE SW - doi 10.1109/MS.2022.3196205 “Can RE Help Better Prepare Industrial AI forCommercial Scale?” By Boris Scharinger, Markus Borg, Andreas Vogelsang, and Thomas Olsson2See “point/counterpoint” discussion provided by Mary Shaw and Liming Zhu - doi 10.1109/MS.2022.3203200 -”Can Software Engineering Harness the Benefits of Advanced AI?”applications in different domains, such as autonomous vehicles, or social media - e.g. chatGPT,or other real projects you are involved in? How is/will RE integrate with policies and (about tocome out) standards?3Workshop Program: The workshop held invited talks in two sessions. Each talk was givenroughly twenty minutes for both presentation and discussion. We were delighted to attractprominent RE for AI researchers from several countries and universities, with diverse experi-ences and opinions. In the following, we list the talk titles, authors, and abstracts.Presenter: Daniel Berry, University of Waterloo, CanadaTitle: “RE for AI: A Hot Topic as Viewed by an RE Alter Kaker”Abstract: I explore how not-very-successful attempts to fit requirements for artificial intelli-gences (AIs) and learned machines (LMs) into the traditional RE mold led to a rethinking aboutRE for AI. I talk about some implications of the rethinking.Presenter: Beatriz Cabrero-Daniel, Chalmers | University of Gothenburg, SwedenTitle: How I learned to stop worrying and discuss about AIAbstract: AI-powered tools, such as autonomous vehicles or chatbots, are increasingly relyingon foundation models. These models are incredibly complex, and training and operating themrequire significant computing resources… which can block academic research, key in evaluatingthe potential harms of such systems and informing about them. There is therefore a need forcollaboration between various societal actors comes in. Authorities, engineers, and ethicistsmust work together to understand the implications of this disruptive AI technology and protectand inform individuals (and society as a whole). The first step to have informed discussionsabout and governance mechanisms for AI is agreeing on the right terms to use. Nevertheless,one might ask oneself whether it is possible to set solid grounds when the AI field is changingso rapidly: we might as well learn to stop worrying and love the AI ;)Presenter: Jaelson Castro, Federal University of Pernambuco, BrazilTitle: Can we trust Socially Assistive Robots? What are their requirements?Abstract: The development and use of Socially Assistive Robots (SARs) has grown significantlyin recent years. Trust is one of the critical aspects for the adoption of robots in a social setting.Trust in Requirements Engineering is considered a non-functional requirement that needs to beproperly satisfied. Hence it needs to be elicited, modeled, validated, and managed. The purposeof this research is to investigate what their requirements are. We report on the challenges,candidate solution paths, and research priorities regarding RE4AI for trustable SARs. Our initialgoal is to develop a catalog of Non-Functional Requirements (NFRs) that is adequate to supportelicitation and specification in projects of socially assistive humanoid robots (SARs), allowingthe identification of possible problems from the point of view of Trust.3For example:. EU AI act https://artificialintelligenceact.eu/; ACM Statement on Principles for Responsible Algo-rithmic Systems www.acm.org/binaries/content/assets/public-policy/final-joint-ai-statement-update.pdf; 7000-2021 -IEEE StandardModel Process for Addressing Ethical Concerns during SystemDesign - https://ieeexplore.ieee.org/doc-ument/9536679Presenter: Xavier Franch, Universitat Politècnica de Catalunya, SpainTitle: A Requirements Engineering Perspective to AI-based Systems DevelopmentAbstract: This talk will reflect on the role that RE should play in the development of AI-basedsystems with a focus on three areas: roles involved, requirements’ scope and non-functionalrequirements. The position taken is that requirements engineers shall become the cornerstonein AI-based system development in collaboration with other key roles.Presenter: Hans-Martin Heyn, Chalmers | University of Gothenburg, SwedenTitle: The challenge of creating an architecture framework for Very Efficient Deep Learning inthe IoTAbstract: The VEDLIoT (Very Efficient Deep Learning in the Internet-of-Things) project hasrun since the beginning of 2021 as a cooperation between six academic and five industrial part-ners from different industry sectors. The aim of the project is to provide tools, methodologies,and experience for supporting the development and deployment of distributed systems withcomponents that rely on deep learning. A key enabler for VEDLIoT is the ability to decomposerequirements and architecture based on the needs of the 13 use cases of the project. Therefore,we propose to extend the state of the art on architecture framework by providing a mathemat-ical model for system architectures, which is scalable and supports co-evolution of differentaspects of an AI system. In this talk we present our work on Requirement Engineering inVEDLIoT and guidelines based on a mathematical formulation on how a consistent architectureframework can be built up that supports the creation and management of system architecturesand requirements for distributed AI systems in VEDLIoT.Presenter: Eric Knauss, Chalmers | University of Gothenburg, SwedenTitle: The Role of Requirements Engineering in the Development of Automotive PerceptionSystemsAbstract: Software that contains machine learning algorithms is an integral part of automotiveperception, for example, in driving automation systems. The development of such software,specifically the training and validation of the machine learning components, require largeannotated datasets. An industry of data and annotation services has emerged to serve thedevelopment of such data-intensive automotive software components. Wide-spread difficul-ties to specify data and annotation needs challenge collaborations between OEMs (OriginalEquipment Manufacturers) and their suppliers of software components, data, and annotations.In this talk, we will describe these challenges from a requirements perspective, including theneed to describe driving scenarios in specific contexts, maintaining the scoping of the contextdescriptions, specifying requirements on data, annotations, and their quality. We describe howthis relates to requirements management and traceability, as well as managing value-chains ofautomotive perception systems across complex customer and supplier relationships. We closethis talk with an outlook on how requirements engineering can support this complex domainand which future research directions we foresee.Presenter: Alejandro Maté, University of Alicante, SpainTitle: Towards a more systematic design of Artificial IntelligenceAbstract: Thanks to its many advantages, Artificial Intelligence (AI) has become a commonpractice for companies and academia. However, AI is far from a simple process where analgorithm is trained and the project is complete. AI requires many steps such as domain under-standing, data preprocessing, model selection, training and evaluation as well as deep knowledgeof a wide variety of traditional and novel algorithms that behave differently. To make thingsworse, most AI projects rarely define clear-cut objectives, often producing failures or larger thanexpected costs when stakeholders notice that the model is less-than-optimal for the real-worlduse. To improve AI practice, we propose to exploit the power of Requirements Engineering. Byclearly describing the objectives pursued together with the qualities of the desired solution suchas explainability or performance with large datasets, we can discard from the initial momentinadequate AI solutions. Even more, an adequate specification of AI Requirements providesthe necessary framework to evaluate the results of the project in a comprehensive way ratherthan just looking at model performance and questioning whether the achieved accuracy washigh enough or not. In order to create the ideal AI Requirements language, we consider thatwe must bridge the gap between the stakeholders and the developers, being understandableenough to specify the high-level objectives while at the same time sufficiently specific to linkthese objectives to the underlying solution.Presenter: Kurt Schneider, Leibniz Universität Hannover, GermanyTitle: Requirements Engineering and Artificial Intelligence in OneAbstract: Many contributions either address RE for AI or AI for RE. In this talk, I will present acase in which both directions occur at the same time.About 10 years ago, machine learning techniques were used to classify bugs, feedback, andeven requirements. We identified security-related requirements using and optimizing Bayesianclassifiers. This was AI4RE, since security requirements could be identified and considered byapplying ML. During the last years, app reviews were classified and interpreted in order toderive change requests and requirements automatically; AI4RE again.The reverse direction (RE4AI) provides performance requirements and constraints for the AIpart of a software. For example, bounds for precision, recall, or accuracy are required. Users,stakeholders, and developers use concepts applicable to the statistical nature of AI and ML toexpress robustness and correctness requirements, e.g. via False Negatives and False Positives.The ultimate goal is to define and phrase requirements on AI in a way that is comprehensible tohumans – and that provides guidance for AI engineers.Explainability is a quality aspect that gains importance as software gets more complex. Ithas two facets: (1) In the artificial intelligence developer community, interpretability is themost relevant variant of explainability. If an ML algorithm can explain how it reached aresult, ML engineers can use that information to improve the algorithms, the training set, orhyperparameters. (2) Software users and stakeholders require certain explanations to be givenunder specific conditions. Often, an explanation should indicate why the software behaved as itdid. However, what are meaningful and realistic requirements on explanations?In the softXplain project, we try to identify reasonable requirements for explainability –no matter whether AI is involved or not. Many people do not know – or do not even care –whether a piece of software contains AI. While using the software, they just need to knowwhat happened, and why, if software behaves in an unexpected way. Explanations may referto traditional software, or it may contain AI/ML components. For example, we need to de-termine when and under which conditions (””triggers””) we want an explanation to be given.One option could be to useML for triggering explanations as required. AI and REmerge into one.Presenter: Renata Guizzardi, University of Twente, the NetherlandsTitle: Dealing with Ethical Requirements for AI Systems: A Requirements Engineering Per-spectiveAbstract: Ethics is a central concern in any civilized society, and we have to take ethicaldecisions everyday. For example, in face of a Pandemic and the consequent shortages in hospitalresources, who should be the first patients to be treated? How to guarantee the fair gradingof students exams when you have a huge class of 300 students and thus require support fromteaching assistants in grading? And so on. Although ethical issues are hard, taking decisionswhen they come up is important and thus, we shouldn’t run from it. Nowadays, more andmore, we are relying on systems (and especially on AI systems) to take decisions on our behalf.So a question that comes to mind is: are these systems being developed to effectively handleethical issues? A look at the literature already suggests they aren’t. Thus, our work has beenfocused on proposing the development of ethical systems by design, claiming this can onlybe done if Requirements Engineering (RE) activities are realized in such a way that ethicalconcerns are focused since the start, and also throughout the whole system’s life cycle. Such anRE method is an open issue, and our first results show that ontologies can be supportive in thiscontext. Using ontologies can help overcome misunderstandings and support the focus on thethings (i.e., domain concepts and relations) that matter. There is still much to be learn by theRE foundations, for instance, on selecting proper stakeholders, representing and formalizingrequirements in different ways etc. Moreover, the important recent works on legal and ethicalframeworks (created by governmental and standard organizations) should be taken into account.And finally, promoting relevant forums like RE4AI, where people working on related topics cangather and share ideas and experiences is paramount to make consistent steps towards ethicalsystems’ development.

Preface:The 4th International Workshop on Requirements Engineering for Artificial Intelligence (RE4AI’23)

https://ris.utwente.nl/ws/files/307259510/RE4AI_preface.pdf

Preface:The 4th International Workshop on Requirements Engineering for Artificial Intelligence (RE4AI’23)

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information