HITECS: A UML Profile and Analysis Framework for Hardware-in-the-Loop Testing of Cyber Physical Systems

Hardware-in-the-loop (HiL) testing is an important step in the development of cyber physical systems (CPS). CPS HiL test cases manipulate hardware components, are time-consuming and their behaviors are impacted by the uncertainties in the CPS environment. To mitigate the risks associated with HiL testing, engineers have to ensure that (1) HiL test cases are well-behaved, i.e., they implement valid test scenarios and do not accidentally damage hardware, and (2) HiL test cases can execute within the time budget allotted to HiL testing. This paper proposes an approach to help engineers systematically specify and analyze CPS HiL test cases. Leveraging the UML profile mechanism, we develop an executable domain-specific language, HITECS, for HiL test case specification. HITECS builds on the UML Testing Profile (UTP) and the UML action language (Alf). Using HITECS, we provide analysis methods to check whether HiL test cases are well-behaved, and to estimate the execution times of these test cases before the actual HiL testing stage. We apply HITECS to an industrial case study from the satellite domain. Our results show that: (1) HITECS is feasible to use in practice; (2) HITECS helps engineers define more complete and effective well-behavedness assertions for HiL test cases, compared to when these assertions are defined without systematic guidance; (3) HITECS verifies in practical time that HiL test cases are well-behaved; and (4) HITECS accurately estimates HiL test case execution times

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Towards a Model-Centric Software Testing Life Cycle for Early and Consistent Testing Activities

The constant improvement of the available computing power nowadays enables the accomplishment of more and more complex tasks. The resulting implicit increase in the complexity of hardware and software solutions for realizing the desired functionality requires a constant improvement of the development methods used. On the one hand over the last decades the percentage of agile development practices, as well as testdriven development increases. On the other hand, this trend results in the need to reduce the complexity with suitable methods. At this point, the concept of abstraction comes into play, which manifests itself in model-based approaches such as MDSD or MBT. The thesis is motivated by the fact that the earliest possible detection and elimination of faults has a significant influence on product costs. Therefore, a holistic approach is developed in the context of model-driven development, which allows applying testing already in early phases and especially on the model artifacts, i.e. it provides a shift left of the testing activities. To comprehensively address the complexity problem, a modelcentric software testing life cycle is developed that maps the process steps and artifacts of classical testing to the model-level. Therefore, the conceptual basis is first created by putting the available model artifacts of all domains into context. In particular, structural mappings are specified across the included domain-specific model artifacts to establish a sufficient basis for all the process steps of the life cycle. Besides, a flexible metamodel including operational semantics is developed, which enables experts to carry out an abstract test execution on the modellevel. Based on this, approaches for test case management, automated test case generation, evaluation of test cases, and quality verification of test cases are developed. In the context of test case management, a mechanism is realized that enables the selection, prioritization, and reduction of Test Model artifacts usable for test case generation. I.e. a targeted set of test cases is generated satisfying quality criteria like coverage at the model-level. These quality requirements are accomplished by using a mutation-based analysis of the identified test cases, which builds on the model basis. As the last step of the model-centered software testing life cycle two approaches are presented, allowing an abstract execution of the test cases in the model context through structural analysis and a form of model interpretation concerning data flow information. All the approaches for accomplishing the problem are placed in the context of related work, as well as examined for their feasibility by of a prototypical implementation within the Architecture And Analysis Framework. Subsequently, the described approaches and their concepts are evaluated by qualitative as well as quantitative evaluation. Moreover, case studies show the practical applicability of the approach

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact. Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases. Topics covered include: Safety Assessment, Reliability Analysis, Critical Systems and Applications, Functional Safety, Dependability Validation, Dependable Software Systems, Embedded Systems, System Certification

Certifications of Critical Systems – The CECRIS Experience

In recent years, a considerable amount of effort has been devoted, both in industry and academia, to the development, validation and verification of critical systems, i.e. those systems whose malfunctions or failures reach a critical level both in terms of risks to human life as well as having a large economic impact.Certifications of Critical Systems – The CECRIS Experience documents the main insights on Cost Effective Verification and Validation processes that were gained during work in the European Research Project CECRIS (acronym for Certification of Critical Systems). The objective of the research was to tackle the challenges of certification by focusing on those aspects that turn out to be more difficult/important for current and future critical systems industry: the effective use of methodologies, processes and tools.The CECRIS project took a step forward in the growing field of development, verification and validation and certification of critical systems. It focused on the more difficult/important aspects of critical system development, verification and validation and certification process. Starting from both the scientific and industrial state of the art methodologies for system development and the impact of their usage on the verification and validation and certification of critical systems, the project aimed at developing strategies and techniques supported by automatic or semi-automatic tools and methods for these activities, setting guidelines to support engineers during the planning of the verification and validation phases

Uncertainty-aware Specification and Analysis for Hardware-in-the-Loop Testing of Cyber Physical Systems

Hardware-in-the-loop (HiL) testing is important for developing cyber physical systems (CPS). HiL test cases manipulate hardware, are time-consuming and their behaviors are impacted by the uncertainties in the CPS environment. To mitigate the risks associated with HiL testing, engineers have to ensure that (1) test cases are well-behaved, e.g., they do not damage hardware, and (2) test cases can execute within a time budget. Leveraging the UML profile mechanism, we develop a domain-specific language, HITECS, for HiL test case specification. Using HITECS, we provide uncertainty-aware analysis methods to check the well-behavedness of HiL test cases. In addition, we provide a method to estimate the execution times of HiL test cases before the actual HiL testing. We apply HITECS to an industrial case study from the satellite domain. Our results show that: (1) HITECS helps engineers define more effective assertions to check HiL test cases, compared to the assertions defined without any systematic guidance; (2) HITECS verifies in practical time that HiL test cases are well-behaved; (3) HITECS is able to resolve uncertain parameters of HiL test cases by synthesizing conditions under which test cases are guaranteed to be well-behaved; and (4) HITECS accurately estimates HiL test case execution times

A Model-Driven Methodology for Critical Systems Engineering

Model-Driven Engineering (MDE) promises to enhance system development by reducing development time, and increasing productivity and quality. MDE is gaining popularity in several industry sectors, and is attractive also for critical systems where they can reduce efforts and costs for verification and validation (V&V), and can ease certification. This thesis proposes a novel model-driven life cycle that is tailored to the development of critical railway systems. It also integrates an original approach for model-driven system validation, based on a new model named Computation Independent Test model (CIT). Moreover, the process supports the Failure Modes and Effect Analysis (FMEA), with a novel approach to conduct Model-Driven FMEA, based on custom SysML Diagram, namely the FMEA Diagram, and Prolog. The approaches have been experimented in multiple real-world case studies, from railway and automative domains

Uncertainty representation in software models: a survey

This paper provides a comprehensive overview and analysis of research work on how uncertainty is currently represented in software models. The survey presents the definitions and current research status of different proposals for addressing uncertainty modeling and introduces a classification framework that allows to compare and classify existing proposals, analyze their current status and identify new trends. In addition, we discuss possible future research directions, opportunities and challenges.This work is partially supported by the European Commission (FEDER) and the Spanish Government under projects APOLO (US1264651), HORATIO (RTI2018-101204-B-C21), EKIPMENT-PLUS (P18-FR-2895) and COSCA (PGC2018-094905-B-I00)

An assurance level sensitive UML profile for supporting DO-178C

Several model-based approaches have been proposed to ease the process of developing certifiable safety-critical software. In this thesis, we are interested in airborne software which must comply with DO-178C standard. However, existing approaches do not provide complete support for all the activities of the software life cycle as defined by DO-178C. In this thesis, we propose an UML profile that captures the concepts of DO-178C and its supplements in order to model the evidence required for certification. This profile provides modeling constructs for the definition of a DO-178C compliant software life cycle, the specification of the software requirements, the specification of verification data and finally the specification of the traceability that is requested by DO-178C. Furthermore, this profile has the unique feature of providing means to specify the objectives and activities to be performed throughout the software life cycle depending on the targeted assurance level and applied DO-178C supplements. We implemented the proposed profile within Papyrus, an UML modeling environment. We used the profile to model a realistic example of airborne software. Specifically, we illustrated the usefulness of the profile through four use cases

Automated validation of minimum risk model-based system designs of complex avionics systems

Today, large civil aircraft incorporate a vast array of complex and coupled subsystems with thousands of electronic control units and software with millions of lines of code. Aircraft suppliers are challenged to provide superior products that are developed at a minimum time and cost, with maximum safety and security. No single person can understand the complex interactions of such a system of systems. Finding an optimal solution from large sets of different possible designs is an impossible task if done manually. Thus, written, non-executable specifications carry a high degree of product uncertainty. As a result, more than two-thirds of all specifications need to be reworked. Since most specification flaws are discovered and resolved at a late stage during development, when expenditures for redesign are at a maximum, the development approach currently used has a high probability of project cost and time overruns or even project failure, thus maximizing the risk of development. It is the aim of this work, to develop a model- and simulation-based systems engineering method with associated design and validation environment that minimizes the risk of development for complex systems, e.g. aircraft. The development risk is a minimum, if all development decisions are validated early against the services of a product at mission level by the final customer. To do so, executable specifications are created during design and validated against the requirements of system services at mission level. Validated executable specifications are used and updated for all decisions from concept development through implementation and training. In addition, virtual prototypes are developed. A virtual prototype is an executable system specification that is combined with human machine interface concept models to include usability requirements in the overall design and to enable interactive specification validation and early end user training by means of interactive user-driven system simulation. In a first step, so called executable workflows and simulation sets are developed to enable the execution of sets of structured and coupled simulation models. In a second step, a model- and simulation-based development and validation process model is developed from concept design to specification development. In a final step, two different validation processes are developed. An automated validation process based on executable specifications and an interactive validation process based on virtual prototypes. For the development of executable specifications and virtual prototypes, plug-and-play capable model components are developed. The developed method is validated for examples from civil aircraft development with focus on avionics and highly configurable and customizable cabin systems.Große zivile Flugzeuge umfassen eine hohe Anzahl von komplexen und gekoppelten Subsystemen mit Tausenden von elektronischen Steuergeräten und Software mit Millionen von Codezeilen. Keine einzelne Person kann die komplexen Wechselwirkungen eines solchen Systems von Systemen verstehen. Daher beinhalten geschriebene, nicht ausführbare Spezifikationen einen hohen Grad an Produktunsicherheit. Infolgedessen müssen mehr als zwei Drittel aller Spezifikationen überarbeitet werden. Da die meisten Spezifikationsfehler zu einem späten Zeitpunkt entdeckt und gelöst werden, wenn Aufwände für Überarbeitungen maximal sind, hat der gegenwärtige Entwicklungsansatz eine hohe Wahrscheinlichkeit für Kosten- und Zeitüberschreitungen oder führt zum Fehlschlagen von Projekten. Hierdurch wird das Entwicklungsrisiko maximiert. Es ist das Ziel dieser Arbeit, eine modell- und simulationsbasierte Entwicklungsmethode mit zugehöriger Entwurfs- und Validierungsumgebung zu entwickeln, welche das Risiko der Entwicklung für komplexe Systeme minimiert. Das Entwicklungsrisiko ist minimal, wenn alle Entwicklungsentscheidungen frühzeitig vom Endkunden gegen die Leistungen eines Produktes auf Missionsebene validiert werden. Dazu werden ausführbare Spezifikationen während des Entwurfs erstellt und anhand der Anforderungen auf Missionsebene validiert. Validierte ausführbare Spezifikationen werden für alle Entscheidungen von der Konzeptentwicklung bis zur Implementierung verwendet und aktualisiert. Darüber hinaus werden virtuelle Prototypen entwickelt, welche ausführbare Spezifikationen mit Konzeptmodellen für Mensch-Maschine-Schnittstellen kombinieren, um Usability-Anforderungen in den Gesamtentwurf aufzunehmen. Dies ermöglicht eine interaktive Validierung sowie frühes Endbenutzertraining mittels benutzergesteuerter Systemsimulation. Es werden ausführbare Arbeitsabläufe und Simulation Sets entwickelt, welche die Ausführung von strukturierten und gekoppelten Simulationsmodellen ermöglichen. Anschließend wird ein modell- und simulationsbasiertes Entwicklungs- und Validierungsprozessmodell vom Konzeptdesign bis zur Spezifikationsentwicklung entwickelt. Hierfür werden zwei verschiedene Validierungsprozesse verwendet. Ein automatisierter Validierungsprozess basierend auf ausführbaren Spezifikationen und ein interaktiver Validierungsprozess basierend auf virtuellen Prototypen. Für die Entwicklung von ausführbaren Spezifikationen und virtuellen Prototypen werden Modellkomponenten entwickelt. Die entwickelte Methode wird mithilfe von Beispielen aus der zivilen Flugzeugentwicklung validiert, insbesondere in Hinblick auf Avionik sowie hoch konfigurierbare und anpassbare Kabinensysteme