Novel Validation Techniques for Autonomous Vehicles

The automotive industry is facing challenges in producing electrical, connected, and autonomous vehicles. Even if these challenges are, from a technical point of view, independent from each other, the market and regulatory bodies require them to be developed and integrated simultaneously. The development of autonomous vehicles implies the development of highly dependable systems. This is a multidisciplinary activity involving knowledge from robotics, computer science, electrical and mechanical engineering, psychology, social studies, and ethics. Nowadays, many Advanced Driver Assistance Systems (ADAS), like Emergency Braking System, Lane Keep Assistant, and Park Assist, are available. Newer luxury cars can drive by themselves on highways or park automatically, but the end goal is to develop completely autonomous driving vehicles, able to go by themselves, without needing human interventions in any situation. The more vehicles become autonomous, the greater the difficulty in keeping them reliable. It enhances the challenges in terms of development processes since their misbehaviors can lead to catastrophic consequences and, differently from the past, there is no more a human driver to mitigate the effects of erroneous behaviors. Primary threats to dependability come from three sources: misuse from the drivers, design systematic errors, and random hardware failures. These safety threats are addressed under various aspects, considering the particular type of item to be designed. In particular, for the sake of this work, we analyze those related to Functional Safety (FuSa), viewed as the ability of a system to react on time and in the proper way to the external environment. From the technological point of view, these behaviors are implemented by electrical and electronic items. Various standards to achieve FuSa have been released over the years. The first, released in 1998, was the IEC 61508. Its last version is the one released in 2010. This standard defines mainly: • a Functional Safety Management System (FSMS); • methods to determine a Safety Integrated Level (SIL); • methods to determine the probability of failures. To adapt the IEC61508 to the automotive industry’s peculiarity, a newer standard, the ISO26262, was released in 2011 then updated in 2018. This standard provides guidelines about FSMS, called in this case Safety Lifecycle, describing how to develop software and hardware components suitable for functional safety. It also provides a different way to compute the SIL, called in this case Automotive SIL (ASIL), allowing us to consider the average driver’s abilities to control the vehicle in case of failures. Moreover, it describes a way to determine the probability of random hardware failures through Failure Mode, Effects, and Diagnostic Analysis (FMEDA). This dissertation contains contributions to three topics: • random hardware failures mitigation; • improvementoftheISO26262HazardAnalysisandRiskAssessment(HARA); • real-time verification of the embedded software. As the main contribution of this dissertation, I address the safety threats due to random hardware failures (RHFs). For this purpose, I propose a novel simulation-based approach to aid the Failure Mode, Effects, and Diagnostic Analysis (FMEDA) required by the ISO26262 standard. Thanks to a SPICE-level model of the item, and the adoption of fault injection techniques, it is possible to simulate its behaviors obtaining useful information to classify the various failure modes. The proposed approach evolved from a mere simulation of the item, allowing only an item-level failure mode classification up to a vehicle-level analysis. The propagation of the failure modes’ effects on the whole vehicle enables us to assess the impacts on the vehicle’s drivability, improving the quality of the classifications. It can be advantageous where it is difficult to predict how the item-level misbehaviors propagate to the vehicle level, as in the case of a virtual differential gear or the mobility system of a robot. It has been chosen since it can be considered similar to the novel light vehicles, such as electric scooters, that are becoming more and more popular. Moreover, my research group has complete access to its design since it is realized by our university’s DIANA students’ team. When a SPICE-level simulation is too long to be performed, or it is not possible to develop a complete model of the item due to intellectual property protection rules, it is possible to aid this process through behavioral models of the item. A simulation of this kind has been performed on a mobile robotic system. Behavioral models of the electronic components were used, alongside mechanical simulations, to assess the software failure mitigation capabilities. Another contribution has been obtained by modifying the main one. The idea was to make it possible to aid also the Hazard Analysis and Risk Assessment (HARA). This assessment is performed during the concept phase, so before starting to design the item implementation. Its goal is to determine the hazards involved in the item functionality and their associated levels of risk. The end goal of this phase is a list of safety goals. For each one of these safety goals, an ASIL has to be determined. Since HARA relies only on designers expertise and knowledge, it lacks in objectivity and repeatability. Thanks to the simulation results, it is possible to predict the effects of the failures on the vehicle’s drivability, allowing us to improve the severity and controllability assessment, thus improving the objectivity. Moreover, since simulation conditions can be stored, it is possible, at any time, to recheck the results and to add new scenarios, improving the repeatability. The third group of contributions is about the real-time verification of embedded software. Through Hardware-In-the-Loop (HIL), a software integration verification has been performed to test a fundamental automotive component, mixed-criticality applications, and multi-agent robots. The first of these contributions is about real-time tests on Body Control Modules (BCM). These modules manage various electronic accessories in the vehicle’s body, like power windows and mirrors, air conditioning, immobilizer, central locking. The main characteristics of BCMs are the communications with other embedded computers via the car’s vehicle bus (Controller Area Network) and to have a high number (hundreds) of low-speed I/Os. As the second contribution, I propose a methodology to assess the error recovery system’s effects on mixed-criticality applications regarding deadline misses. The system runs two tasks: a critical airplane longitudinal control and a non-critical image compression algorithm. I start by presenting the approach on a benchmark application containing an instrumented bug into the lower criticality task; then, we improved it by injecting random errors inside the lower criticality task’s memory space through a debugger. In the latter case, thanks to the HIL, it is possible to pause the time domain simulation when the debugger operates and resume it once the injection is complete. In this way, it is possible to interact with the target without interfering with the simulation results, combining a full control of the target with an accurate time-domain assessment. The last contribution of this third group is about a methodology to verify, on multi-agent robots, the synchronization between two agents in charge to move the end effector of a delta robot: the correct position and speed of the end effector at any time is strongly affected by a loss of synchronization. The last two contributions may seem unrelated to the automotive industry, but interest in these applications is gaining. Mixed-criticality systems allow reducing the number of ECUs inside cars (for cost reduction), while the multi-agent approach is helpful to improve the cooperation of the connected cars with respect to other vehicles and the infrastructure. The fourth contribution, contained in the appendix, is about a machine learning application to improve the social acceptance of autonomous vehicles. The idea is to improve the comfort of the passengers by recognizing their emotions. I started with the idea to modify the vehicle’s driving style based on a real-time emotions recognition system but, due to the difficulties of performing such operations in an experimental setup, I move to analyze them offline. The emotions are determined on volunteers’ facial expressions recorded while viewing 3D representa- tions showing different calibrations. Thanks to the passengers’ emotional responses, it is possible to choose the better calibration from the comfort point of view

Novel Validation Techniques for Autonomous Vehicles

L'abstract è presente nell'allegato / the abstract is in the attachmen

System software techniques to enhance reliability of modern platforms

192

A generic framework for process execution and secure multi-party transaction authorization

Process execution engines are not only an integral part of workflow and business process management systems but are increasingly used to build process-driven applications. In other words, they are potentially used in all kinds of software across all application domains. However, contemporary process engines and workflow systems are unsuitable for use in such diverse application scenarios for several reasons. The main shortcomings can be observed in the areas of interoperability, versatility, and programmability. Therefore, this thesis makes a step away from domain specific, monolithic workflow engines towards generic and versatile process runtime frameworks, which enable integration of process technology into all kinds of software. To achieve this, the idea and corresponding architecture of a generic and embeddable process virtual machine (ePVM), which supports defining process flows along the theoretical foundation of communicating extended finite state machines, are presented. The architecture focuses on the core process functionality such as control flow and state management, monitoring, persistence, and communication, while using JavaScript as a process definition language. This approach leads to a very generic yet easily programmable process framework. A fully functional prototype implementation of the proposed framework is provided along with multiple example applications. Despite the fact that business processes are increasingly automated and controlled by information systems, humans are still involved, directly or indirectly, in many of them. Thus, for process flows involving sensitive transactions, a highly secure authorization scheme supporting asynchronous multi-party transaction authorization must be available within process management systems. Therefore, along with the ePVM framework, this thesis presents a novel approach for secure remote multi-party transaction authentication - the zone trusted information channel (ZTIC). The ZTIC approach uniquely combines multiple desirable properties such as the highest level of security, ease-of-use, mobility, remote administration, and smooth integration with existing infrastructures into one device and method. Extensively evaluating both, the ePVM framework and the ZTIC, this thesis shows that ePVM in combination with the ZTIC approach represents a unique and very powerful framework for building workflow systems and process-driven applications including support for secure multi-party transaction authorization

INTRUSION DETECTION OF A SIMULATED SCADA SYSTEM USING A DATA-DRIVEN MODELING APPROACH

Supervisory Control and Data Acquisition (SCADA) are large, geographically distributed systems that regulate help processes in industries such as nuclear power, transportation or manufacturing. SCADA is a combination of physical, sensing, and communications equipment that is used for monitoring, control and telemetry acquisition actions. Because SCADA often control the distribution of vital resources such as electricity and water, there is a need to protect these cyber-physical systems from those with possible malicious intent. To this end, an Intrusion Detection System (IDS) is utilized to monitor telemetry sources in order to detect unwanted activities and maintain overall system integrity. This dissertation presents the results in developing a behavior-based approach to intrusion detection using a simulated SCADA test bed. Empirical modeling techniques known as Auto Associative Kernel Regression (AAKR) and Auto Associative Multivariate State Estimation Technique (AAMSET) are used to learn the normal behavior of the test bed. The test bed was then subjected to repeated intrusion injection experiments using penetration testing software and exploit codes. Residuals generated from these experiments are then supplied to an anomaly detection algorithm known as the Sequential Probability Ratio Test (SPRT). This approach is considered novel in that the AAKR and AAMSET, combined with the SPRT, have not been utilized previously in industry for cybersecurity purposes. Also presented in this dissertation is a newly developed variable grouping algorithm that is based on the Auto Correlation Function (ACF) for a given set of input data. Variable grouping is needed for these modeling methods to arrive at a suitable set of predictors that return the lowest error in model performance. The developed behavior-based techniques were able to successfully detect many types of intrusions that include network reconnaissance, DoS, unauthorized access, and information theft. These methods would then be useful in detecting unwanted activities of intruders from both inside and outside of the monitored network. These developed methods would also serve to add an additional layer of security. When compared with two separate variable grouping methods, the newly developed grouping method presented in this dissertation was shown to extract similar groups or groups with lower average model prediction errors

Design of a secure architecture for the exchange of biomedical information in m-Health scenarios

El paradigma de m-Salud (salud móvil) aboga por la integración masiva de las más avanzadas tecnologías de comunicación, red móvil y sensores en aplicaciones y sistemas de salud, para fomentar el despliegue de un nuevo modelo de atención clínica centrada en el usuario/paciente. Este modelo tiene por objetivos el empoderamiento de los usuarios en la gestión de su propia salud (p.ej. aumentando sus conocimientos, promocionando estilos de vida saludable y previniendo enfermedades), la prestación de una mejor tele-asistencia sanitaria en el hogar para ancianos y pacientes crónicos y una notable disminución del gasto de los Sistemas de Salud gracias a la reducción del número y la duración de las hospitalizaciones. No obstante, estas ventajas, atribuidas a las aplicaciones de m-Salud, suelen venir acompañadas del requisito de un alto grado de disponibilidad de la información biomédica de sus usuarios para garantizar una alta calidad de servicio, p.ej. fusionar varias señales de un usuario para obtener un diagnóstico más preciso. La consecuencia negativa de cumplir esta demanda es el aumento directo de las superficies potencialmente vulnerables a ataques, lo que sitúa a la seguridad (y a la privacidad) del modelo de m-Salud como factor crítico para su éxito. Como requisito no funcional de las aplicaciones de m-Salud, la seguridad ha recibido menos atención que otros requisitos técnicos que eran más urgentes en etapas de desarrollo previas, tales como la robustez, la eficiencia, la interoperabilidad o la usabilidad. Otro factor importante que ha contribuido a retrasar la implementación de políticas de seguridad sólidas es que garantizar un determinado nivel de seguridad implica unos costes que pueden ser muy relevantes en varias dimensiones, en especial en la económica (p.ej. sobrecostes por la inclusión de hardware extra para la autenticación de usuarios), en el rendimiento (p.ej. reducción de la eficiencia y de la interoperabilidad debido a la integración de elementos de seguridad) y en la usabilidad (p.ej. configuración más complicada de dispositivos y aplicaciones de salud debido a las nuevas opciones de seguridad). Por tanto, las soluciones de seguridad que persigan satisfacer a todos los actores del contexto de m-Salud (usuarios, pacientes, personal médico, personal técnico, legisladores, fabricantes de dispositivos y equipos, etc.) deben ser robustas y al mismo tiempo minimizar sus costes asociados. Esta Tesis detalla una propuesta de seguridad, compuesta por cuatro grandes bloques interconectados, para dotar de seguridad a las arquitecturas de m-Salud con unos costes reducidos. El primer bloque define un esquema global que proporciona unos niveles de seguridad e interoperabilidad acordes con las características de las distintas aplicaciones de m-Salud. Este esquema está compuesto por tres capas diferenciadas, diseñadas a la medidas de los dominios de m-Salud y de sus restricciones, incluyendo medidas de seguridad adecuadas para la defensa contra las amenazas asociadas a sus aplicaciones de m-Salud. El segundo bloque establece la extensión de seguridad de aquellos protocolos estándar que permiten la adquisición, el intercambio y/o la administración de información biomédica -- por tanto, usados por muchas aplicaciones de m-Salud -- pero no reúnen los niveles de seguridad detallados en el esquema previo. Estas extensiones se concretan para los estándares biomédicos ISO/IEEE 11073 PHD y SCP-ECG. El tercer bloque propone nuevas formas de fortalecer la seguridad de los tests biomédicos, que constituyen el elemento esencial de muchas aplicaciones de m-Salud de carácter clínico, mediante codificaciones novedosas. Finalmente el cuarto bloque, que se sitúa en paralelo a los anteriores, selecciona herramientas genéricas de seguridad (elementos de autenticación y criptográficos) cuya integración en los otros bloques resulta idónea, y desarrolla nuevas herramientas de seguridad, basadas en señal -- embedding y keytagging --, para reforzar la protección de los test biomédicos.The paradigm of m-Health (mobile health) advocates for the massive integration of advanced mobile communications, network and sensor technologies in healthcare applications and systems to foster the deployment of a new, user/patient-centered healthcare model enabling the empowerment of users in the management of their health (e.g. by increasing their health literacy, promoting healthy lifestyles and the prevention of diseases), a better home-based healthcare delivery for elderly and chronic patients and important savings for healthcare systems due to the reduction of hospitalizations in number and duration. It is a fact that many m-Health applications demand high availability of biomedical information from their users (for further accurate analysis, e.g. by fusion of various signals) to guarantee high quality of service, which on the other hand entails increasing the potential surfaces for attacks. Therefore, it is not surprising that security (and privacy) is commonly included among the most important barriers for the success of m-Health. As a non-functional requirement for m-Health applications, security has received less attention than other technical issues that were more pressing at earlier development stages, such as reliability, eficiency, interoperability or usability. Another fact that has contributed to delaying the enforcement of robust security policies is that guaranteeing a certain security level implies costs that can be very relevant and that span along diferent dimensions. These include budgeting (e.g. the demand of extra hardware for user authentication), performance (e.g. lower eficiency and interoperability due to the addition of security elements) and usability (e.g. cumbersome configuration of devices and applications due to security options). Therefore, security solutions that aim to satisfy all the stakeholders in the m-Health context (users/patients, medical staff, technical staff, systems and devices manufacturers, regulators, etc.) shall be robust and, at the same time, minimize their associated costs. This Thesis details a proposal, composed of four interrelated blocks, to integrate appropriate levels of security in m-Health architectures in a cost-efcient manner. The first block designes a global scheme that provides different security and interoperability levels accordingto how critical are the m-Health applications to be implemented. This consists ofthree layers tailored to the m-Health domains and their constraints, whose security countermeasures defend against the threats of their associated m-Health applications. Next, the second block addresses the security extension of those standard protocols that enable the acquisition, exchange and/or management of biomedical information | thus, used by many m-Health applications | but do not meet the security levels described in the former scheme. These extensions are materialized for the biomedical standards ISO/IEEE 11073 PHD and SCP-ECG. Then, the third block proposes new ways of enhancing the security of biomedical standards, which are the centerpiece of many clinical m-Health applications, by means of novel codings. Finally the fourth block, with is parallel to the others, selects generic security methods (for user authentication and cryptographic protection) whose integration in the other blocks results optimal, and also develops novel signal-based methods (embedding and keytagging) for strengthening the security of biomedical tests. The layer-based extensions of the standards ISO/IEEE 11073 PHD and SCP-ECG can be considered as robust, cost-eficient and respectful with their original features and contents. The former adds no attributes to its data information model, four new frames to the service model |and extends four with new sub-frames|, and only one new sub-state to the communication model. Furthermore, a lightweight architecture consisting of a personal health device mounting a 9 MHz processor and an aggregator mounting a 1 GHz processor is enough to transmit a 3-lead electrocardiogram in real-time implementing the top security layer. The extra requirements associated to this extension are an initial configuration of the health device and the aggregator, tokens for identification/authentication of users if these devices are to be shared and the implementation of certain IHE profiles in the aggregator to enable the integration of measurements in healthcare systems. As regards to the extension of SCP-ECG, it only adds a new section with selected security elements and syntax in order to protect the rest of file contents and provide proper role-based access control. The overhead introduced in the protected SCP-ECG is typically 2{13 % of the regular file size, and the extra delays to protect a newly generated SCP-ECG file and to access it for interpretation are respectively a 2{10 % and a 5 % of the regular delays. As regards to the signal-based security techniques developed, the embedding method is the basis for the proposal of a generic coding for tests composed of biomedical signals, periodic measurements and contextual information. This has been adjusted and evaluated with electrocardiogram and electroencephalogram-based tests, proving the objective clinical quality of the coded tests, the capacity of the coding-access system to operate in real-time (overall delays of 2 s for electrocardiograms and 3.3 s for electroencephalograms) and its high usability. Despite of the embedding of security and metadata to enable m-Health services, the compression ratios obtained by this coding range from ' 3 in real-time transmission to ' 5 in offline operation. Complementarily, keytagging permits associating information to images (and other signals) by means of keys in a secure and non-distorting fashion, which has been availed to implement security measures such as image authentication, integrity control and location of tampered areas, private captioning with role-based access control, traceability and copyright protection. The tests conducted indicate a remarkable robustness-capacity tradeoff that permits implementing all this measures simultaneously, and the compatibility of keytagging with JPEG2000 compression, maintaining this tradeoff while setting the overall keytagging delay in only ' 120 ms for any image size | evidencing the scalability of this technique. As a general conclusion, it has been demonstrated and illustrated with examples that there are various, complementary and structured manners to contribute in the implementation of suitable security levels for m-Health architectures with a moderate cost in budget, performance, interoperability and usability. The m-Health landscape is evolving permanently along all their dimensions, and this Thesis aims to do so with its security. Furthermore, the lessons learned herein may offer further guidance for the elaboration of more comprehensive and updated security schemes, for the extension of other biomedical standards featuring low emphasis on security or privacy, and for the improvement of the state of the art regarding signal-based protection methods and applications

Langage dédié et analyse automatisée pour la détection de patrons au sein de traces d'exécution

RÉSUMÉ La complexité des systèmes informatiques distribués et à plusieurs unités de calcul a introduit de nouvelles classes de problèmes. Ces problèmes sont difficiles à reproduire et leur complexité accrue a suggéré l'introduction de nouvelles méthodes de compréhension des systèmes. L'analyse dynamique, à l'aide de traces d'exécution, permet de comprendre les systèmes à partir de leurs données d'exécution. Les traces d'exécution enregistrent, sous forme d'événements, les informations précises et détaillées de l'exécution du système instrumenté. Pour des systèmes comme le noyau d'exploitation de Linux, le traçage fournit des événements de bas niveau (appels systèmes, fautes de pages). De plus, en quelques secondes, le fichier de trace peut enregistrer des millions d'événements. Des visionneuses de trace, telle que Trace Compass, ont été développées dans le but de fournir des analyses de la trace sous différents angles de vue tels que l’allocation des ressources ou l’usage des unités de calcul, et à un haut niveau d'abstraction. Cependant, au-delà des analyses déjà fournies par les visionneuses, les utilisateurs souhaiteraient pouvoir créer des analyses personnalisées qui représenteraient mieux leurs besoins. Par exemple, un utilisateur pourrait tenter de vérifier si le système a subi une attaque particulière. Il faudrait dans ce cas précis pouvoir appliquer à la trace une analyse spécialisée qui permettrait de vérifier la présence d’une séquence d’événements ou d’informations qui décrit l’attaque recherchée. Il existe donc un besoin quant à la nécessité d'identifier des formes particulières ou de retrouver des séquences d'intérêts au sein de la trace. Ce travail propose l'introduction d'un langage déclaratif utilisant les automates finis pour la description de patrons d'intérêts. Les patrons décrits sont ensuite passés à un analyseur élaboré afin de vérifier et repérer leurs présences au sein de traces d’exécution. L'utilisation de machines à états pour la description des patrons permet de décrire des patrons complexes. Ainsi, la mise en place d'un mécanisme de suivi de l'exécution des patrons a été réalisée. Le langage déclaratif proposé est conçu de façon déclarative en XML. Il permet de représenter avec succès tous les types de patrons rencontrés dans la littérature (patrons de détection d'attaques, patrons de test de programmes, patrons d'évaluation de performance, patrons d'agrégations des événements...). La spécification du langage proposé permet de créer des événements synthétiques qui peuvent être traités par l'analyseur au fur et à mesure qu'ils sont créés. La solution proposée dans ce mémoire devrait être capable de traiter les traces de grandes tailles (500MB et plus); la performance en terme de temps d'extraction des données est donc importante. Nous nous assurons qu'elle est au moins aussi bonne que celle des travaux antérieurs du même genre et que la déviation par rapport à la méthode d'extraction standard de Trace Compass reste acceptable. La solution proposée écrit directement les données sur le disque. Elle n’accumule donc pas d’informations en mémoire. L’analyse en terme d’espace en mémoire est donc négligeable. De plus, nous démontrons l'utilité de l'approche proposée à l'aide d'exemples concrets de cas d'utilisation. Une tentative de découverte de la source d'un défaut de requête Web est présentée ainsi qu'un exemple de détection d'attaque dans le système. Enfin, nous proposons à la fin de notre étude une liste de suggestions pour des améliorations possibles à la solution en termes de description des patrons et de réduction du temps de l’analyse.----------ABSTRACT The complexity of distributed and multi-core systems introduced new classes of problems. These problems could be difficult to reproduce and their increasing complexity has suggested the introduction of new methods of systems comprehension. Dynamic analysis, through execution traces, allows to understand the system behavior from its execution data. The execution traces record, in the form of events, accurate and detailed information about the execution of an instrumented system. For systems like the Linux kernel, tracing provides low level events such as system calls and page faults. Moreover, in a few seconds, the trace file can record millions of events. Visualizers, like Trace Compass, were developed to provide analysis of the trace from different points of view such as ressources allocation and CPU usage, and with a high-level of abstraction. However, beyond the analyses that have been already provided by the visualizers, users would like to be able to create custom analyses that better represent their needs. For example, a user could attempt to verify if the system is under a particular attack. It should then be possible to apply to the trace a specialized analysis that would verify the presence of a sequence of events or information that describes the intended attack. Thus, there is a need to be able to identify patterns or to find predefined sequences of events within the trace. This work proposes the introduction of a declarative automata-based pattern description language. The described patterns are then passed to an analyzer designed to efficiently verify and detect their presence within the execution trace. The use of state machines allows to describe complex patterns. Thus, a mechanism to follow the execution of the patterns has been implemented. The proposed language is designed declaratively in XML. It has successfully represented all the types of pattern found in the literature (security attack patterns, testing patterns, system performance patterns, events aggregation patterns, ...). The language specification allows to create synthetic events that can be processed by the analyzer as they are created. This proposal should be able to process large trace files (1GB or more). Thus, performance in terms of time of data extraction is important. We ensure that this solution is at least as good as previous ones of the same kind and that the deviation from the standard extraction method of Trace Compass remains acceptable. The proposed solution writes the data directly to disk. It therefore does not accumulate information in memory. The analysis in terms of memory space is negligible. In addition, we try to demonstrate the usefulness of our approach with some application test cases. An example of an attempt to find the root cause of a web request defect and an example of attack detection in a system are presented. Finally, we propose at the end of this study a list of suggestions for possible improvements of the solution in terms of the description of patterns and reduction of the time of the analysis