210 research outputs found

    Evaluating Architectural Safeguards for Uncertain AI Black-Box Components

    Get PDF
    Although tremendous progress has been made in Artificial Intelligence (AI), it entails new challenges. The growing complexity of learning tasks requires more complex AI components, which increasingly exhibit unreliable behaviour. In this book, we present a model-driven approach to model architectural safeguards for AI components and analyse their effect on the overall system reliability

    Chatbots for Modelling, Modelling of Chatbots

    Full text link
    Tesis Doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de Lectura: 28-03-202

    Software Architecture in Practice: Challenges and Opportunities

    Full text link
    Software architecture has been an active research field for nearly four decades, in which previous studies make significant progress such as creating methods and techniques and building tools to support software architecture practice. Despite past efforts, we have little understanding of how practitioners perform software architecture related activities, and what challenges they face. Through interviews with 32 practitioners from 21 organizations across three continents, we identified challenges that practitioners face in software architecture practice during software development and maintenance. We reported on common software architecture activities at software requirements, design, construction and testing, and maintenance stages, as well as corresponding challenges. Our study uncovers that most of these challenges center around management, documentation, tooling and process, and collects recommendations to address these challenges.Comment: Preprint of Full Research Paper, the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '23

    CodePlan: Repository-level Coding using LLMs and Planning

    Full text link
    Software engineering activities such as package migration, fixing errors reports from static analysis or testing, and adding type annotations or other specifications to a codebase, involve pervasively editing the entire repository of code. We formulate these activities as repository-level coding tasks. Recent tools like GitHub Copilot, which are powered by Large Language Models (LLMs), have succeeded in offering high-quality solutions to localized coding problems. Repository-level coding tasks are more involved and cannot be solved directly using LLMs, since code within a repository is inter-dependent and the entire repository may be too large to fit into the prompt. We frame repository-level coding as a planning problem and present a task-agnostic framework, called CodePlan to solve it. CodePlan synthesizes a multi-step chain of edits (plan), where each step results in a call to an LLM on a code location with context derived from the entire repository, previous code changes and task-specific instructions. CodePlan is based on a novel combination of an incremental dependency analysis, a change may-impact analysis and an adaptive planning algorithm. We evaluate the effectiveness of CodePlan on two repository-level tasks: package migration (C#) and temporal code edits (Python). Each task is evaluated on multiple code repositories, each of which requires inter-dependent changes to many files (between 2-97 files). Coding tasks of this level of complexity have not been automated using LLMs before. Our results show that CodePlan has better match with the ground truth compared to baselines. CodePlan is able to get 5/6 repositories to pass the validity checks (e.g., to build without errors and make correct code edits) whereas the baselines (without planning but with the same type of contextual information as CodePlan) cannot get any of the repositories to pass them

    Rule-based Graph Repair using Minimally Restricted Consistency-Improving Transformations

    Full text link
    Model-driven software engineering is a suitable method for dealing with the ever-increasing complexity of software development processes. Graphs and graph transformations have proven useful for representing such models and changes to them. These models must satisfy certain sets of constraints. An example are the multiplicities of a class structure. During the development process, a change to a model may result in an inconsistent model that must at some point be repaired. This problem is called model repair. In particular, we will consider rule-based graph repair which is defined as follows: Given a graph GG, a constraint cc such that GG does not satisfy cc, and a set of rules RR, use the rules of R\mathcal{R} to transform GG into a graph that satisfies cc. Known notions of consistency have either viewed consistency as a binary property, either a graph is consistent w.r.t. a constraint cc or not, or only viewed the number of violations of the first graph of a constraint. In this thesis, we introduce new notions of consistency, which we call consistency-maintaining and consistency-increasing transformations and rules, respectively. This is based on the possibility that a constraint can be satisfied up to a certain nesting level. We present constructions for direct consistency-maintaining or direct consistency-increasing application conditions, respectively. Finally, we present an rule-based graph repair approach that is able to repair so-called \emph{circular conflict-free constraints}, and so-called circular conflict-free sets of constraints. Intuitively, a set of constraint CC is circular conflict free, if there is an ordering c1,,cnc_1, \ldots, c_n of all constraints of CC such that there is no j<ij <i such that a repair of cic_i at all graphs satisfying cjc_j leads to a graph not satisfying cjc_j

    Evaluating Architectural Safeguards for Uncertain AI Black-Box Components

    Get PDF
    Künstliche Intelligenz (KI) hat in den vergangenen Jahren große Erfolge erzielt und ist immer stärker in den Fokus geraten. Insbesondere Methoden des Deep Learning (ein Teilgebiet der KI), in dem Tiefe Neuronale Netze (TNN) zum Einsatz kommen, haben beeindruckende Ergebnisse erzielt, z.B. im autonomen Fahren oder der Mensch-Roboter-Interaktion. Die immense Datenabhängigkeit und Komplexität von TNN haben jedoch gravierende Schwachstellen offenbart. So reagieren TNN sensitiv auf bestimmte Einflussfaktoren der Umwelt (z.B. Helligkeits- oder Kontraständerungen in Bildern) und führen zu falschen Vorhersagen. Da KI (und insbesondere TNN) in sicherheitskritischen Systemen eingesetzt werden, kann solch ein Verhalten zu lebensbedrohlichen Situationen führen. Folglich haben sich neue Forschungspotenziale entwickelt, die sich explizit der Absicherung von KI-Verfahren widmen. Ein wesentliches Problem bei vielen KI-Verfahren besteht darin, dass ihr Verhalten oder Vorhersagen auf Grund ihrer hohen Komplexität nicht erklärt bzw. nachvollzogen werden können. Solche KI-Modelle werden auch als Black-Box bezeichnet. Bestehende Arbeiten adressieren dieses Problem, in dem zur Laufzeit “bösartige” Eingabedaten identifiziert oder auf Basis von Ein- und Ausgaben potenziell falsche Vorhersagen erkannt werden. Arbeiten in diesem Bereich erlauben es zwar potenziell unsichere Zustände zu erkennen, machen allerdings keine Aussagen, inwiefern mit solchen Situationen umzugehen ist. Somit haben sich eine Reihe von Ansätzen auf Architektur- bzw. Systemebene etabliert, um mit KI-induzierten Unsicherheiten umzugehen (z.B. N-Version-Programming-Muster oder Simplex Architekturen). Darüber hinaus wächst die Anforderung an KI-basierte Systeme sich zur Laufzeit anzupassen, um mit sich verändernden Bedingungen der Umwelt umgehen zu können. Systeme mit solchen Fähigkeiten sind bekannt als Selbst-Adaptive Systeme. Software-Ingenieure stehen nun vor der Herausforderung, aus einer Menge von Architekturellen Sicherheitsmechanismen, den Ansatz zu identifizieren, der die nicht-funktionalen Anforderungen bestmöglich erfüllt. Jeder Ansatz hat jedoch unterschiedliche Auswirkungen auf die Qualitätsattribute des Systems. Architekturelle Entwurfsentscheidungen gilt es so früh wie möglich (d.h. zur Entwurfszeit) aufzulösen, um nach der Implementierung des Systems Änderungen zu vermeiden, die mit hohen Kosten verbunden sind. Darüber hinaus müssen insbesondere sicherheitskritische Systeme den strengen (Qualitäts-) Anforderungen gerecht werden, die bereits auf Architektur-Ebene des Software-Systems adressiert werden müssen. Diese Arbeit befasst sich mit einem modellbasierten Ansatz, der Software-Ingenieure bei der Entwicklung von KI-basierten System unterstützt, um architekturelle Entwurfsentscheidungen (bzw. architekturellen Sicherheitsmechanismen) zum Umgang mit KI-induzierten Unsicherheiten zu bewerten. Insbesondere wird eine Methode zur Zuverlässigkeitsvorhersage von KI-basierten Systemen auf Basis von etablierten modellbasierten Techniken erforscht. In einem weiteren Schritt wird die Erweiterbarkeit/Verallgemeinerbarkeit der Zuverlässigkeitsvorhersage für Selbst-Adaptive Systeme betrachtet. Der Kern beider Ansätze ist ein Umweltmodell zur Modellierung () von KI-spezifischen Unsicherheiten und () der operativen Umwelt des Selbst-Adaptiven Systems. Zuletzt wird eine Klassifikationsstruktur bzw. Taxonomie vorgestellt, welche, auf Basis von verschiedenen Dimensionen, KI-basierte Systeme in unterschiedliche Klassen einteilt. Jede Klasse ist mit einem bestimmten Grad an Verlässlichkeitszusicherungen assoziiert, die für das gegebene System gemacht werden können. Die Dissertation umfasst vier zentrale Beiträge. 1. Domänenunabhängige Modellierung von KI-spezifischen Umwelten: In diesem Beitrag wurde ein Metamodell zur Modellierung von KI-spezifischen Unsicherheiten und ihrer zeitlichen Ausdehnung entwickelt, welche die operative Umgebung eines selbstadaptiven Systems bilden. 2. Zuverlässigkeitsvorhersage von KI-basierten Systemen: Der vorgestellte Ansatz erweitert eine existierende Architekturbeschreibungssprache (genauer: Palladio Component Model) zur Modellierung von Komponenten-basierten Software-Architekturen sowie einem dazugehörigenWerkzeug zur Zuverlässigkeitsvorhersage (für klassische Software-Systeme). Das Problem der Black-Box-Eigenschaft einer KI-Komponente wird durch ein Sensitivitätsmodell adressiert, das, in Abhängigkeit zu verschiedenen Unsicherheitsfaktoren, die Prädektive Unsicherheit einer KI-Komponente modelliert. 3. Evaluation von Selbst-Adaptiven Systemen: Dieser Beitrag befasst sich mit einem Rahmenwerk für die Evaluation von Selbst-Adaptiven Systemen, welche für die Absicherung von KI-Komponenten vorgesehen sind. Die Arbeiten zu diesem Beitrag verallgemeinern/erweitern die Konzepte von Beitrag 2 für Selbst-Adaptive Systeme. 4. Klassen der Verlässlichkeitszusicherungen: Der Beitrag beschreibt eine Klassifikationsstruktur, die den Grad der Zusicherung (in Bezug auf bestimmte Systemeigenschaften) eines KI-basierten Systems bewertet. Der zweite Beitrag wurde im Rahmen einer Fallstudie aus dem Bereich des Autonomen Fahrens validiert. Es wurde geprüft, ob Plausibilitätseigenschaften bei der Zuverlässigkeitsvorhersage erhalten bleiben. Hierbei konnte nicht nur die Plausibilität des Ansatzes nachgewiesen werden, sondern auch die generelle Möglichkeit Entwurfsentscheidungen zur Entwurfszeit zu bewerten. Für die Validierung des dritten Beitrags wurden ebenfalls Plausibilitätseigenschaften geprüft (im Rahmen der eben genannten Fallstudie und einer Fallstudie aus dem Bereich der Mensch-Roboter-Interaktion). Darüber hinaus wurden zwei weitere Community-Fallstudien betrachtet, bei denen (auf Basis von Simulatoren) Selbst-Adaptive Systeme bewertet und mit den Ergebnissen unseres Ansatzes verglichen wurden. In beiden Fällen konnte gezeigt werden, dass zum einen alle Plausibilitätseigenschaft erhalten werden und zum anderen, der Ansatz dieselben Ergebnisse erzeugt, wie die Domänen-spezifischen Simulatoren. Darüber hinaus konnten wir zeigen, dass unser Ansatz Software-Ingenieure bzgl. der Bewertung von Entwurfsentscheidungen, die für die Entwicklung von Selbst-Adaptiven Systemen relevant sind, unterstützt. Der erste Beitrag wurde implizit mit Beitrag 2 und mit 3 validiert. Für den vierten Beitrag wurde die Klassifikationsstruktur auf bekannte und repräsentative KI-Systeme angewandt und diskutiert. Es konnte jedes KI-System in eine der Klassen eingeordnet werden, so dass die generelle Anwendbarkeit der Klassifikationsstruktur gezeigt wurde

    Understanding, Analysis, and Handling of Software Architecture Erosion

    Get PDF
    Architecture erosion occurs when a software system's implemented architecture diverges from the intended architecture over time. Studies show erosion impacts development, maintenance, and evolution since it accumulates imperceptibly. Identifying early symptoms like architectural smells enables managing erosion through refactoring. However, research lacks comprehensive understanding of erosion, unclear which symptoms are most common, and lacks detection methods. This thesis establishes an erosion landscape, investigates symptoms, and proposes identification approaches. A mapping study covers erosion definitions, symptoms, causes, and consequences. Key findings: 1) "Architecture erosion" is the most used term, with four perspectives on definitions and respective symptom types. 2) Technical and non-technical reasons contribute to erosion, negatively impacting quality attributes. Practitioners can advocate addressing erosion to prevent failures. 3) Detection and correction approaches are categorized, with consistency and evolution-based approaches commonly mentioned.An empirical study explores practitioner perspectives through communities, surveys, and interviews. Findings reveal associated practices like code review and tools identify symptoms, while collected measures address erosion during implementation. Studying code review comments analyzes erosion in practice. One study reveals architectural violations, duplicate functionality, and cyclic dependencies are most frequent. Symptoms decreased over time, indicating increased stability. Most were addressed after review. A second study explores violation symptoms in four projects, identifying 10 categories. Refactoring and removing code address most violations, while some are disregarded.Machine learning classifiers using pre-trained word embeddings identify violation symptoms from code reviews. Key findings: 1) SVM with word2vec achieved highest performance. 2) fastText embeddings worked well. 3) 200-dimensional embeddings outperformed 100/300-dimensional. 4) Ensemble classifier improved performance. 5) Practitioners found results valuable, confirming potential.An automated recommendation system identifies qualified reviewers for violations using similarity detection on file paths and comments. Experiments show common methods perform well, outperforming a baseline approach. Sampling techniques impact recommendation performance

    Risk Analysis for Smart Cities Urban Planners: Safety and Security in Public Spaces

    Get PDF
    Christopher Alexander in his famous writings "The Timeless Way of Building" and "A pattern language" defined a formal language for the description of a city. Alexander developed a generative grammar able to formally describe complex and articulated concepts of architecture and urban planning to define a common language that would facilitate both the participation of ordinary citizens and the collaboration between professionals in architectural and urban planning. In this research, a similar approach has been applied to let two domains communicate although they are very far in terms of lexicon, methodologies and objectives. These domains are urban planning, urban design and architecture, seen as the first domain both in terms of time and in terms of completeness of vision, and the one relating to the world of engineering, made by innumerable disciplines. In practice, there is a domain that defines the requirements and the overall vision (the first) and a domain (the second) which implements them with real infrastructures and systems. To put these two worlds seamlessly into communication, allowing the concepts of the first world to be translated into those of the second, Christopher Alexander’s idea has been followed by defining a common language. By applying Essence, the software engineering formal descriptive theory, using its customization rules, to the concept of a Smart City, a common language to completely trace the requirements at all levels has been defined. Since the focus was on risk analysis for safety and security in public spaces, existing risk models have been considered, evidencing a further gap also within the engineering world itself. Depending on the area being considered, risk management models have different and siloed approaches which ignore the interactions of one type of risk with the others. To allow effective communication between the two domains and within the engineering domain, a unified risk analysis framework has been developed. Then a framework (an ontology) capable of describing all the elements of a Smart City has been developed and combined with the common language to trace the requirements. Following the philosophy of the Vienna Circle, a creative process called Aufbau has then been defined to allow the generation of a detailed description of the Smart City, at any level, using the common language and the ontology above defined. Then, the risk analysis methodology has been applied to the city model produced by Aufbau. The research developed tools to apply such results to the entire life cycle of the Smart City. With these tools, it is possible to understand how much a given architectural, urban planning or urban design requirement is operational at a given moment. In this way, the narration can accurately describe how much the initial requirements set by architects, planners and urban designers and, above all, the values required by stakeholders, are satisfied, at any time. The impact of this research on urban planning is the ability to create a single model between the two worlds, leaving everyone free to express creativity and expertise in the appropriate forms but, at the same time, allowing both to fill the communication gap existing today. This new way of planning requires adequate IT tools and takes the form, from the engineering side, of harmonization of techniques already in use and greater clarity of objectives. On the side of architecture, urban planning and urban design, it is instead a powerful decision support tool, both in the planning and operational phases. This decision support tool for Urban Planning, based on the research results, is the starting point for the development of a meta-heuristic process using an evolutionary approach. Consequently, risk management, from Architecture/Urban Planning/Urban Design up to Engineering, in any phase of the Smart City’s life cycle, is seen as an “organism” that evolves.Christopher Alexander nei suoi famosi scritti "The Timeless Way of Building" e "A pattern language" ha definito un linguaggio formale per la descrizione di una città, sviluppando una grammatica in grado di descrivere formalmente concetti complessi e articolati di architettura e urbanistica, definendo un linguaggio comune per facilitare la partecipazione dei comuni cittadini e la collaborazione tra professionisti. In questa ricerca, un approccio simile è stato applicato per far dialogare due domini sebbene siano molto distanti in termini di lessico, metodologie e obiettivi. Essi sono l'urbanistica, l'urban design e l'architettura, visti come primo dominio sia in termini di tempo che di completezza di visione, e quello del mondo dell'ingegneria, con numerose discipline. In pratica, esiste un dominio che definisce i requisiti e la visione d'insieme (il primo) e un dominio (il secondo) che li implementa con infrastrutture e sistemi reali. Per metterli in perfetta comunicazione, permettendo di tradurre i concetti del primo in quelli del secondo, si è seguita l'idea di Alexander definendo un linguaggio. Applicando Essence, la teoria descrittiva formale dell'ingegneria del software al concetto di Smart City, è stato definito un linguaggio comune per tracciarne i requisiti a tutti i livelli. Essendo il focus l'analisi dei rischi per la sicurezza negli spazi pubblici, sono stati considerati i modelli di rischio esistenti, evidenziando un'ulteriore lacuna anche all'interno del mondo dell'ingegneria stessa. A seconda dell'area considerata, i modelli di gestione del rischio hanno approcci diversi e isolati che ignorano le interazioni di un tipo di rischio con gli altri. Per consentire una comunicazione efficace tra i due domini e all'interno del dominio dell'ingegneria, è stato sviluppato un quadro di analisi del rischio unificato. Quindi è stato sviluppato un framework (un'ontologia) in grado di descrivere tutti gli elementi di una Smart City e combinato con il linguaggio comune per tracciarne i requisiti. Seguendo la filosofia del Circolo di Vienna, è stato poi definito un processo creativo chiamato Aufbau per consentire la generazione di una descrizione dettagliata della Smart City, a qualsiasi livello, utilizzando il linguaggio comune e l'ontologia sopra definita. Infine, la metodologia dell'analisi del rischio è stata applicata al modello di città prodotto da Aufbau. La ricerca ha sviluppato strumenti per applicare tali risultati all'intero ciclo di vita della Smart City. Con questi strumenti è possibile capire quanto una data esigenza architettonica, urbanistica o urbanistica sia operativa in un dato momento. In questo modo, la narrazione può descrivere con precisione quanto i requisiti iniziali posti da architetti, pianificatori e urbanisti e, soprattutto, i valori richiesti dagli stakeholder, siano soddisfatti, in ogni momento. L'impatto di questa ricerca sull'urbanistica è la capacità di creare un modello unico tra i due mondi, lasciando ognuno libero di esprimere creatività e competenza nelle forme appropriate ma, allo stesso tempo, permettendo ad entrambi di colmare il gap comunicativo oggi esistente. Questo nuovo modo di progettare richiede strumenti informatici adeguati e si concretizza, dal lato ingegneristico, in un'armonizzazione delle tecniche già in uso e in una maggiore chiarezza degli obiettivi. Sul versante dell'architettura, dell'urbanistica e del disegno urbano, è invece un potente strumento di supporto alle decisioni, sia in fase progettuale che operativa. Questo strumento di supporto alle decisioni per la pianificazione urbana, basato sui risultati della ricerca, è il punto di partenza per lo sviluppo di un processo meta-euristico utilizzando un approccio evolutivo

    LASSO – an observatorium for the dynamic selection, analysis and comparison of software

    Full text link
    Mining software repositories at the scale of 'big code' (i.e., big data) is a challenging activity. As well as finding a suitable software corpus and making it programmatically accessible through an index or database, researchers and practitioners have to establish an efficient analysis infrastructure and precisely define the metrics and data extraction approaches to be applied. Moreover, for analysis results to be generalisable, these tasks have to be applied at a large enough scale to have statistical significance, and if they are to be repeatable, the artefacts need to be carefully maintained and curated over time. Today, however, a lot of this work is still performed by human beings on a case-by-case basis, with the level of effort involved often having a significant negative impact on the generalisability and repeatability of studies, and thus on their overall scientific value. The general purpose, 'code mining' repositories and infrastructures that have emerged in recent years represent a significant step forward because they automate many software mining tasks at an ultra-large scale and allow researchers and practitioners to focus on defining the questions they would like to explore at an abstract level. However, they are currently limited to static analysis and data extraction techniques, and thus cannot support (i.e., help automate) any studies which involve the execution of software systems. This includes experimental validations of techniques and tools that hypothesise about the behaviour (i.e., semantics) of software, or data analysis and extraction techniques that aim to measure dynamic properties of software. In this thesis a platform called LASSO (Large-Scale Software Observatorium) is introduced that overcomes this limitation by automating the collection of dynamic (i.e., execution-based) information about software alongside static information. It features a single, ultra-large scale corpus of executable software systems created by amalgamating existing Open Source software repositories and a dedicated DSL for defining abstract selection and analysis pipelines. Its key innovations are integrated capabilities for searching for selecting software systems based on their exhibited behaviour and an 'arena' that allows their responses to software tests to be compared in a purely data-driven way. We call the platform a 'software observatorium' since it is a place where the behaviour of large numbers of software systems can be observed, analysed and compared

    Технология комплексной поддержки жизненного цикла семантически совместимых интеллектуальных компьютерных систем нового поколения

    Get PDF
    В издании представлено описание текущей версии открытой технологии онтологического проектирования, производства и эксплуатации семантически совместимых гибридных интеллектуальных компьютерных систем (Технологии OSTIS). Предложена стандартизация интеллектуальных компьютерных систем, а также стандартизация методов и средств их проектирования, что является важнейшим фактором, обеспечивающим семантическую совместимость интеллектуальных компьютерных систем и их компонентов, что существенное снижение трудоемкости разработки таких систем. Книга предназначена всем, кто интересуется проблемами искусственного интеллекта, а также специалистам в области интеллектуальных компьютерных систем и инженерии знаний. Может быть использована студентами, магистрантами и аспирантами специальности «Искусственный интеллект». Табл. 8. Ил. 223. Библиогр.: 665 назв
    corecore