Hacking an Ambiguity Detection Tool to Extract Variation Points: an Experience Report

Natural language (NL) requirements documents can be a precious source to identify variability information. This information can be later used to define feature models from which different systems can be instantiated. In this paper, we are interested in validating the approach we have recently proposed to extract variability issues from the ambiguity defects found in NL requirement documents. To this end, we single out ambiguities using an available NL analysis tool, QuARS, and we classify the ambiguities returned by the tool by distinguishing among false positives, real ambiguities, and variation points. We consider three medium sized requirement documents from different domains, namely, train control, social web, home automation. We report in this paper the results of the assessment. Although the validation set is not so large, the results obtained are quite uniform and permit to draw some interesting conclusions. Starting from the results obtained, we can foresee the tailoring of a NL analysis tool for extracting variability from NL requirement documents

Consistency-Preserving Evolution Planning on Feature Models

A software product line (SPL) enables large-scale reuse in a family of related software systems through configurable features. SPLs represent a long-term investment so that their ongoing evolution becomes paramount and requires careful planning. While existing approaches enable to create an evolution plan for an SPL on feature-model (FM) level, they assume the plan to be rigid and do not support retroactive changes. In this paper, we present a method that enables to create and retroactively adapt an FM evolution plan while preventing undesired impacts on its structural and logical consistency. This method is founded in structural operational semantics and linear temporal logic. We implement our method using rewriting logic, integrate it within an FM tool suite and perform an evaluation using a collection of existing FM evolution scenarios

Empirical assessment of generating adversarial configurations for software product lines

Software product line (SPL) engineering allows the derivation of products tailored to stakeholders’ needs through the setting of a large number of configuration options. Unfortunately, options and their interactions create a huge configuration space which is either intractable or too costly to explore exhaustively. Instead of covering all products, machine learning (ML) approximates the set of acceptable products (e.g., successful builds, passing tests) out of a training set (a sample of configurations). However, ML techniques can make prediction errors yielding non-acceptable products wasting time, energy and other resources. We apply adversarial machine learning techniques to the world of SPLs and craft new configurations faking to be acceptable configurations but that are not and vice-versa. It allows to diagnose prediction errors and take appropriate actions. We develop two adversarial configuration generators on top of state-of-the-art attack algorithms and capable of synthesizing configurations that are both adversarial and conform to logical constraints. We empirically assess our generators within two case studies: an industrial video synthesizer (MOTIV) and an industry-strength, open-source Web-app configurator (JHipster). For the two cases, our attacks yield (up to) a 100% misclassification rate without sacrificing the logical validity of adversarial configurations. This work lays the foundations of a quality assurance framework for ML-based SPLs

Empirical Assessment of Generating Adversarial Configurations for Software Product Lines

International audienceSoftware product line (SPL) engineering allows the derivation of products tailored to stakeholders' needs through the setting of a large number of configuration options. Unfortunately, options and their interactions create a huge configuration space which is either intractable or too costly to explore exhaustively. Instead of covering all products, machine learning (ML) approximates the set of acceptable products (e.g., successful builds, passing tests) out of a training set (a sample of configurations). However, ML techniques can make prediction errors yielding non-acceptable products wasting time, energy and other resources. We apply adversarial machine learning techniques to the world of SPLs and craft new configurations faking to be acceptable configurations but that are not and vice-versa. It allows to diagnose prediction errors and take appropriate actions. We develop two adversarial configuration generators on top of state-of-the-art attack algorithms and capable of synthesizing configurations that are both adversarial and conform to logical constraints. We empirically assess our generators within two case studies: an industrial video synthesizer (MOTIV) and an industry-strength, open-source Web-appconfigurator (JHipster). For the two cases, our attacks yield (up to) a 100% misclassification rate without sacrificing the logical validity of adversarial configurations. This work lays the foundations of a quality assurance framework for ML-based SPLs

Konsistente Feature Modell gesteuerte Softwareproduktlinien Evolution

SPLs are an approach to manage families of closely related software systems in terms of configurable functionality. A feature model captures common and variable functionalities of an SPL on a conceptual level in terms of features. Reusable artifacts, such as code, documentation, or tests are related to features using a feature-artifact mapping. A product of an SPL can be derived by selecting features in a configuration. Over the course of time, SPLs and their artifacts are subject to change. As SPLs are particularly complex, their evolution is a challenging task. Consequently, SPL evolution must be thoroughly planned well in advance. However, plans typically do not turn out as expected and, thus, replanning is required. Feature models lean themselves for driving SPL evolution. However, replanning of feature-model evolution can lead to inconsistencies and feature-model anomalies may be introduced during evolution. Along with feature-model evolution, other SPL artifacts, especially configurations, need to consistently evolve. The work of this thesis provides remedy to the aforementioned challenges by presenting an approach for consistent evolution of SPLs. The main contributions of this thesis can be distinguished into three key areas: planning and replanning feature-model evolution, analyzing feature-model evolution, and consistent SPL artifact evolution. As a starting point for SPL evolution, we introduce Temporal Feature Models (TFMs) that allow capturing the entire evolution timeline of a feature model in one artifact, i.e., past history, present changes, and planned evolution steps. We provide an execution semantics of feature-model evolution operations that guarantees consistency of feature-model evolution timelines. To keep feature models free from anomalies, we introduce analyses to detect anomalies in feature-model evolution timelines and explain these anomalies in terms of their causing evolution operations. To enable consistent SPL artifact evolution, we generalize the concept of modeling evolution timelines in TFMs to be applicable for any modeling language. Moreover, we provide a methodology that enables involved engineers to define and use guidance for configuration evolution.Softwareproduktlinien (SPLs) ermöglichen es, konfigurierbare Funktionalität von eng verwandten Softwaresystemen zu verwalten. In einem Feature Modell werden gemeinsame und variable Funktionalitäten einer SPL auf Basis abstrakter Features modelliert. Wiederverwendbare Artefakte werden in einem Feature-Artefakt Mapping Features zugeordnet. Ein Produkt einer SPL kann abgeleitet werden, indem Features in einer Konfiguration ausgewählt werden. Im Laufe der Zeit müssen sich SPLs und deren Artefakte verändern. Da SPLs ganze Softwarefamilien modellieren, ist deren Evolution eine besonders herausfordernde Aufgabe, die gründlich im Voraus geplant werden muss. Feature Modelle eignen sich besonders als Planungsmittel einer SPL. Umplanung von Feature Modell Evolution kann jedoch zu Inkonsistenzen führen und Feature Modell Anomalien können im Zuge der Evolution eingeführt werden. Im Anschluss an die Feature Modell Evolution muss die Evolution anderer SPL Artefakte, insbesondere Konfigurationen, konsistent modelliert werden. In dieser Arbeit wird ein Ansatz zur konsistenten Evolution von SPLs vorgestellt, der die zuvor genannten Herausforderungen adressiert. Die Beiträge dieser Arbeit lassen sich in drei Kernbereiche aufteilen: Planung und Umplanung von Feature Modell Evolution, Analyse von Feature Modell Evolution und konsistente Evolution von SPL Artefakten. Temporal Feature Models (TFMs) werden als Startpunkt für SPL Evolution eingeführt. In einem TFM wird die gesamte Evolutionszeitlinie eines Feature Modells in einem Artefakt abgebildet, was sowohl vergangene Änderungen, den aktuellen Zustand, als auch geplante Änderungen beinhaltet. Auf Basis einer Ausführungssemantik wird die Konsistenz von Feature Modell Evolutionszeitlinien sichergestellt. Um Feature Modelle frei von Anomalien zu halten, werden Analysen eingeführt, welche die gesamte Evolutionszeitlinie eines Feature Modells auf Anomalien untersucht und diese mit verursachenden Evolutionsoperationen erklärt. Das Konzept zur Modellierung von Feature Modell Evolutionszeitlinien aus TFMs wird verallgemeinert, um die gesamte Evolution von Modellen beliebiger Modellierungssprachen spezifizieren zu können. Des Weiteren wird eine Methodik vorgestellt, die beteiligten Ingenieuren eine geführte Evolution von Konfigurationen ermöglicht

Modelling, Reverse Engineering, and Learning Software Variability

The society expects software to deliver the right functionality, in a short amount of time and with fewer resources, in every possible circumstance whatever are the hardware, the operating systems, the compilers, or the data fed as input. For fitting such a diversity of needs, it is common that software comes in many variants and is highly configurable through configuration options, runtime parameters, conditional compilation directives, menu preferences, configuration files, plugins, etc. As there is no one-size-fits-all solution, software variability ("the ability of a software system or artifact to be efficiently extended, changed, customized or configured for use in a particular context") has been studied the last two decades and is a discipline of its own. Though highly desirable, software variability also introduces an enormous complexity due to the combinatorial explosion of possible variants. For example, the Linux kernel has 15000+ options and most of them can have 3 values: "yes", "no", or "module". Variability is challenging for maintaining, verifying, and configuring software systems (Web applications, Web browsers, video tools, etc.). It is also a source of opportunities to better understand a domain, create reusable artefacts, deploy performance-wise optimal systems, or find specialized solutions to many kinds of problems. In many scenarios, a model of variability is either beneficial or mandatory to explore, observe, and reason about the space of possible variants. For instance, without a variability model, it is impossible to establish a sampling strategy that would satisfy the constraints among options and meet coverage or testing criteria. I address a central question in this HDR manuscript: How to model software variability? I detail several contributions related to modelling, reverse engineering, and learning software variability. I first contribute to support the persons in charge of manually specifying feature models, the de facto standard for modeling variability. I develop an algebra together with a language for supporting the composition, decomposition, diff, refactoring, and reasoning of feature models. I further establish the syntactic and semantic relationships between feature models and product comparison matrices, a large class of tabular data. I then empirically investigate how these feature models can be used to test in the large configurable systems with different sampling strategies. Along this effort, I report on the attempts and lessons learned when defining the "right" variability language. From a reverse engineering perspective, I contribute to synthesize variability information into models and from various kinds of artefacts. I develop foundations and methods for reverse engineering feature models from satisfiability formulae, product comparison matrices, dependencies files and architectural information, and from Web configurators. I also report on the degree of automation and show that the involvement of developers and domain experts is beneficial to obtain high-quality models. Thirdly, I contribute to learning constraints and non-functional properties (performance) of a variability-intensive system. I describe a systematic process "sampling, measuring, learning" that aims to enforce or augment a variability model, capturing variability knowledge that domain experts can hardly express. I show that supervised, statistical machine learning can be used to synthesize rules or build prediction models in an accurate and interpretable way. This process can even be applied to huge configuration space, such as the Linux kernel one. Despite a wide applicability and observed benefits, I show that each individual line of contributions has limitations. I defend the following answer: a supervised, iterative process (1) based on the combination of reverse engineering, modelling, and learning techniques; (2) capable of integrating multiple variability information (eg expert knowledge, legacy artefacts, dynamic observations). Finally, this work opens different perspectives related to so-called deep software variability, security, smart build of configurations, and (threats to) science

Improving data preparation for the application of process mining

Immersed in what is already known as the fourth industrial revolution, automation and data exchange are taking on a particularly relevant role in complex environments, such as industrial manufacturing environments or logistics. This digitisation and transition to the Industry 4.0 paradigm is causing experts to start analysing business processes from other perspectives. Consequently, where management and business intelligence used to dominate, process mining appears as a link, trying to build a bridge between both disciplines to unite and improve them. This new perspective on process analysis helps to improve strategic decision making and competitive capabilities. Process mining brings together data and process perspectives in a single discipline that covers the entire spectrum of process management. Through process mining, and based on observations of their actual operations, organisations can understand the state of their operations, detect deviations, and improve their performance based on what they observe. In this way, process mining is an ally, occupying a large part of current academic and industrial research. However, although this discipline is receiving more and more attention, it presents severe application problems when it is implemented in real environments. The variety of input data in terms of form, content, semantics, and levels of abstraction makes the execution of process mining tasks in industry an iterative, tedious, and manual process, requiring multidisciplinary experts with extensive knowledge of the domain, process management, and data processing. Currently, although there are numerous academic proposals, there are no industrial solutions capable of automating these tasks. For this reason, in this thesis by compendium we address the problem of improving business processes in complex environments thanks to the study of the state-of-the-art and a set of proposals that improve relevant aspects in the life cycle of processes, from the creation of logs, log preparation, process quality assessment, and improvement of business processes. Firstly, for this thesis, a systematic study of the literature was carried out in order to gain an in-depth knowledge of the state-of-the-art in this field, as well as the different challenges faced by this discipline. This in-depth analysis has allowed us to detect a number of challenges that have not been addressed or received insufficient attention, of which three have been selected and presented as the objectives of this thesis. The first challenge is related to the assessment of the quality of input data, known as event logs, since the requeriment of the application of techniques for improving the event log must be based on the level of quality of the initial data, which is why this thesis presents a methodology and a set of metrics that support the expert in selecting which technique to apply to the data according to the quality estimation at each moment, another challenge obtained as a result of our analysis of the literature. Likewise, the use of a set of metrics to evaluate the quality of the resulting process models is also proposed, with the aim of assessing whether improvement in the quality of the input data has a direct impact on the final results. The second challenge identified is the need to improve the input data used in the analysis of business processes. As in any data-driven discipline, the quality of the results strongly depends on the quality of the input data, so the second challenge to be addressed is the improvement of the preparation of event logs. The contribution in this area is the application of natural language processing techniques to relabel activities from textual descriptions of process activities, as well as the application of clustering techniques to help simplify the results, generating more understandable models from a human point of view. Finally, the third challenge detected is related to the process optimisation, so we contribute with an approach for the optimisation of resources associated with business processes, which, through the inclusion of decision-making in the creation of flexible processes, enables significant cost reductions. Furthermore, all the proposals made in this thesis are validated and designed in collaboration with experts from different fields of industry and have been evaluated through real case studies in public and private projects in collaboration with the aeronautical industry and the logistics sector

Model-Based Engineering of Collaborative Embedded Systems

This Open Access book presents the results of the "Collaborative Embedded Systems" (CrESt) project, aimed at adapting and complementing the methodology underlying modeling techniques developed to cope with the challenges of the dynamic structures of collaborative embedded systems (CESs) based on the SPES development methodology. In order to manage the high complexity of the individual systems and the dynamically formed interaction structures at runtime, advanced and powerful development methods are required that extend the current state of the art in the development of embedded systems and cyber-physical systems. The methodological contributions of the project support the effective and efficient development of CESs in dynamic and uncertain contexts, with special emphasis on the reliability and variability of individual systems and the creation of networks of such systems at runtime. The project was funded by the German Federal Ministry of Education and Research (BMBF), and the case studies are therefore selected from areas that are highly relevant for Germany’s economy (automotive, industrial production, power generation, and robotics). It also supports the digitalization of complex and transformable industrial plants in the context of the German government's "Industry 4.0" initiative, and the project results provide a solid foundation for implementing the German government's high-tech strategy "Innovations for Germany" in the coming years