Testbench qualification of SystemC TLM protocols through Mutation Analysis

Transaction-level modeling (TLM) has become the de-facto reference modeling style for system-level design and verification of embedded systems. It allows designers to implement high-level communication protocols for simulations up to 1000x faster than at register-transfer level (RTL). To guarantee interoperability between TLM IP suppliers and users, designers implement the TLM communication protocols by relying on a reference standard, such as the standard OSCI for SystemC TLM. Functional correctness of such protocols as well as their compliance to the reference TLM standard are usually verified through user-defined testbenches, which high-quality and completeness play a key role for an efficient TLM design and verification flow. This article presents a methodology to apply mutation analysis, a technique applied in literature for SW testing, for measuring the testbench quality in verifying TLM protocols. In particular, the methodology aims at (i) qualifying the testbenches by considering both the TLM protocol correctness and their compliance to a defined standard (i.e., OSCI TLM), (ii) optimizing the simulation time during mutation analysis by avoiding mutation redundancies, and (iii) driving the designers in the testbench improvement. Experimental results on benchmarks of different complexity and architectural characteristics are reported to analyze the methodology applicability

A Cross-level Verification Methodology for Digital IPs Augmented with Embedded Timing Monitors

Smart systems are characterized by the integration in a single device of multi-domain subsystems of different technological domains, namely, analog, digital, discrete and power devices, MEMS, and power sources. Such challenges, emerging from the heterogeneous nature of the whole system, combined with the traditional challenges of digital design, directly impact on performance and on propagation delay of digital components. This article proposes a design approach to enhance the RTL model of a given digital component for the integration in smart systems with the automatic insertion of delay sensors, which can detect and correct timing failures. The article then proposes a methodology to verify such added features at system level. The augmented model is abstracted to SystemC TLM, which is automatically injected with mutants (i.e., code mutations) to emulate delays and timing failures. The resulting TLM model is finally simulated to identify timing failures and to verify the correctness of the inserted delay monitors. Experimental results demonstrate the applicability of the proposed design and verification methodology, thanks to an efficient sensor-aware abstraction methodology, by applying the flow to three complex case studies

A Cross-level Verification Methodology for Digital IPs Augmented with Embedded Timing Monitors

Smart systems implement the leading technology advances in the context of embedded devices. Current design methodologies are not suitable to deal with tightly interacting subsystems of different technological domains, namely analog, digital, discrete and power devices, MEMS and power sources. The interaction effects between the components and between the environment and the system must be modeled and simulated at system level to achieve high performance. Focusing on digital subsystem, additional design constraints have to be considered as a result of the integration of multi-domain subsystems in a single device. The main digital design challenges combined with those emerging from the heterogeneous nature of the whole system directly impact on performance, hence propagation delay, of the digital component. In this paper we propose a design approach to enhance the RTL model of a given digital component for the integration in smart systems, and a methodology to verify the added features at system-level. The design approach consists of ``augmenting'' the RTL model through the automatic insertion of delay sensors, which are capable of detecting and correcting timing failures. The verification methodology consists of an automatic flow of two steps. Firstly the augmented model is abstracted to system-level (i.e., SystemC TLM); secondly mutants, which are code mutations to emulate timing failures, are automatically injected into the abstracted model. Experimental results demonstrate the applicability of the proposed design and verification methodology and the effectiveness of the simulation performance

A SystemC-based Platform for Assertion-based Verification and Mutation Analysis in Systems Biology

Boolean models are gaining an increasing interest for reproducing dynamic behaviours, understanding processes, and predicting emerging properties of cellular signalling networks through in-silico experiments. They are emerging as avalid alternative to the quantitative approaches (i.e., based on ordinary differential equations) for exploratory modelling when little is known about reaction kinetics or equilibrium constants in the context of gene expression or signalling. Even though several approaches and software have been recently proposed for logic modelling of biological systems, they are limited to specific modelling contexts and they lack of automation in analysing biological properties such as complex attractors, molecule vulnerability, dose response. This paper presents a design and verification platform based on SystemC that applies methodologies and tools well established in the electronic-design automation (EDA) fieldsuch as assertion-based verification (ABV) and mutation analysis, which allow complex attractors (i.e., protein oscillations) and robustness/sensitivity of the signalling networks to be simulated and analysed. The paper reports the results obtained by applying such verification techniques for the analysis of the intracellular signalling network controlling integrin activation mediating leukocyte recruitment from the blood into the tissues

Multi-Domain Fault Models Covering the Analog Side of a Smart or Cyber-Physical System

Over the last decade, the industrial world has been involved in a massive revolution guided by the adoption of digital technologies. In this context, complex systems like cyber-physical systems play a fundamental role since they were designed and realized by composing heterogeneous components. The combined simulation of the behavioral models of these components allows to reproduce the nominal behavior of the real system. Similarly, a smart system is a device that integrates heterogeneous components but in a miniaturized form factor. The development of smart or cyber-physical systems, in combination with faulty behaviors modeled for the different physical domains composing the system, enables to support advanced functional safety assessment at the system level. A methodology to create and inject multi-domain fault models in the analog side of these systems has been proposed by exploiting the physical analogy between the electrical and mechanical domains to infer a new mechanical fault taxonomy. Thus, standard electrical fault models are injected into the electrical part, while the derived mechanical fault models are injected directly into the mechanical part. The entire flow has been applied to two case studies: a direct current motor connected with a gear train, and a three-axis accelerometer

Re-use of tests and arguments for assesing dependable mixed-critically systems

The safety assessment of mixed-criticality systems (MCS) is a challenging activity due to system heterogeneity, design constraints and increasing complexity. The foundation for MCSs is the integrated architecture paradigm, where a compact hardware comprises multiple execution platforms and communication interfaces to implement concurrent functions with different safety requirements. Besides a computing platform providing adequate isolation and fault tolerance mechanism, the development of an MCS application shall also comply with the guidelines defined by the safety standards. A way to lower the overall MCS certification cost is to adopt a platform-based design (PBD) development approach. PBD is a model-based development (MBD) approach, where separate models of logic, hardware and deployment support the analysis of the resulting system properties and behaviour. The PBD development of MCSs benefits from a composition of modular safety properties (e.g. modular safety cases), which support the derivation of mixed-criticality product lines. The validation and verification (V&V) activities claim a substantial effort during the development of programmable electronics for safety-critical applications. As for the MCS dependability assessment, the purpose of the V&V is to provide evidences supporting the safety claims. The model-based development of MCSs adds more V&V tasks, because additional analysis (e.g., simulations) need to be carried out during the design phase. During the MCS integration phase, typically hardware-in-the-loop (HiL) plant simulators support the V&V campaigns, where test automation and fault-injection are the key to test repeatability and thorough exercise of the safety mechanisms. This dissertation proposes several V&V artefacts re-use strategies to perform an early verification at system level for a distributed MCS, artefacts that later would be reused up to the final stages in the development process: a test code re-use to verify the fault-tolerance mechanisms on a functional model of the system combined with a non-intrusive software fault-injection, a model to X-in-the-loop (XiL) and code-to-XiL re-use to provide models of the plant and distributed embedded nodes suited to the HiL simulator, and finally, an argumentation framework to support the automated composition and staged completion of modular safety-cases for dependability assessment, in the context of the platform-based development of mixed-criticality systems relying on the DREAMS harmonized platform.La dificultad para evaluar la seguridad de los sistemas de criticidad mixta (SCM) aumenta con la heterogeneidad del sistema, las restricciones de diseño y una complejidad creciente. Los SCM adoptan el paradigma de arquitectura integrada, donde un hardware embebido compacto comprende múltiples plataformas de ejecución e interfaces de comunicación para implementar funciones concurrentes y con diferentes requisitos de seguridad. Además de una plataforma de computación que provea un aislamiento y mecanismos de tolerancia a fallos adecuados, el desarrollo de una aplicación SCM además debe cumplir con las directrices definidas por las normas de seguridad. Una forma de reducir el coste global de la certificación de un SCM es adoptar un enfoque de desarrollo basado en plataforma (DBP). DBP es un enfoque de desarrollo basado en modelos (DBM), en el que modelos separados de lógica, hardware y despliegue soportan el análisis de las propiedades y el comportamiento emergente del sistema diseñado. El desarrollo DBP de SCMs se beneficia de una composición modular de propiedades de seguridad (por ejemplo, casos de seguridad modulares), que facilitan la definición de líneas de productos de criticidad mixta. Las actividades de verificación y validación (V&V) representan un esfuerzo sustancial durante el desarrollo de aplicaciones basadas en electrónica confiable. En la evaluación de la seguridad de un SCM el propósito de las actividades de V&V es obtener las evidencias que apoyen las aseveraciones de seguridad. El desarrollo basado en modelos de un SCM incrementa las tareas de V&V, porque permite realizar análisis adicionales (por ejemplo, simulaciones) durante la fase de diseño. En las campañas de pruebas de integración de un SCM habitualmente se emplean simuladores de planta hardware-in-the-loop (HiL), en donde la automatización de pruebas y la inyección de faltas son la clave para la repetitividad de las pruebas y para ejercitar completamente los mecanismos de tolerancia a fallos. Esta tesis propone diversas estrategias de reutilización de artefactos de V&V para la verificación temprana de un MCS distribuido, artefactos que se emplearán en ulteriores fases del desarrollo: la reutilización de código de prueba para verificar los mecanismos de tolerancia a fallos sobre un modelo funcional del sistema combinado con una inyección de fallos de software no intrusiva, la reutilización de modelo a X-in-the-loop (XiL) y código a XiL para obtener modelos de planta y nodos distribuidos aptos para el simulador HiL y, finalmente, un marco de argumentación para la composición automatizada y la compleción escalonada de casos de seguridad modulares, en el contexto del desarrollo basado en plataformas de sistemas de criticidad mixta empleando la plataforma armonizada DREAMS.Kritikotasun nahastuko sistemen segurtasun ebaluazioa jarduera neketsua da beraien heterogeneotasuna dela eta. Sistema hauen oinarria arkitektura integratuen paradigman datza, non hardware konpaktu batek exekuzio plataforma eta komunikazio interfaze ugari integratu ahal dituen segurtasun baldintza desberdineko funtzio konkurrenteak inplementatzeko. Konputazio plataformek isolamendu eta akatsen aurkako mekanismo egokiak emateaz gain, segurtasun arauek definituriko jarraibideak jarraitu behar dituzte kritikotasun mistodun aplikazioen garapenean. Sistema hauen zertifikazio prozesuaren kostua murrizteko aukera bat plataformetan oinarritutako garapenean (PBD) datza. Garapen planteamendu hau modeloetan oinarrituriko garapena da (MBD) non modeloaren logika, hardware eta garapen desberdinak sistemaren propietateen eta portaeraren aurka aztertzen diren. Kritikotasun mistodun sistemen PBD garapenak etekina ateratzen dio moduluetan oinarrituriko segurtasun propietateei, adibidez: segurtasun kasu modularrak (MSC). Modulu hauek kritikotasun mistodun produktu-lerroak ere hartzen dituzte kontutan. Berifikazio eta balioztatze (V&V) jarduerek esfortzu kontsideragarria eskatzen dute segurtasun-kiritikoetarako elektronika programagarrien garapenean. Kritikotasun mistodun sistemen konfiantzaren ebaluazioaren eta V&V jardueren helburua segurtasun eskariak jasotzen dituzten frogak proportzionatzea da. Kritikotasun mistodun sistemen modelo bidezko garapenek zeregin gehigarriak atxikitzen dizkio V&V jarduerari, fase honetan analisi gehigarriak (hots, simulazioak) zehazten direlako. Bestalde, kritikotasun mistodun sistemen integrazio fasean, hardware-in-the-loop (Hil) simulazio plantek V&V iniziatibak sostengatzen dituzte non testen automatizazioan eta akatsen txertaketan funtsezko jarduerak diren. Jarduera hauek frogen errepikapena eta segurtasun mekanismoak egiaztzea ahalbidetzen dute. Tesi honek V&V artefaktuen berrerabilpenerako estrategiak proposatzen ditu, kritikotasun mistodun sistemen egiaztatze azkarrerako sistema mailan eta garapen prozesuko azken faseetaraino erabili daitezkeenak. Esate baterako, test kodearen berrabilpena akats aurkako mekanismoak egiaztatzeko, modelotik X-in-the-loop (XiL)-ra eta kodetik XiL-rako konbertsioa HiL simulaziorako eta argumentazio egitura bat DREAMS Europear proiektuan definituriko arkitektura estiloan oinarrituriko segurtasun kasu modularrak automatikoki eta gradualki sortzeko

Modeling and Simulation of Biological Systems through Electronic Design Automation techniques

Modeling and simulation of biological systems is a key requirement for integrating invitro and in-vivo experimental data. In-silico simulation allows testing different experimental conditions, thus helping in the discovery of the dynamics that regulate the system. These dynamics include errors in the cellular information processing that are responsible for diseases such as cancer, autoimmunity, and diabetes as well as drug effects to the system (Gonalves, 2013). In this context, modeling approaches can be classified into two categories: quantitative and qualitative models. Quantitative modeling allows for a natural representation of molecular and gene networks and provides the most precise prediction. Nevertheless, the lack of kinetic data (and of quantitative data in general) hampers its use for many situations (Le Novere, 2015). In contrast, qualitative models simplify the biological reality and are often able to reproduce the system behavior. They cannot describe actual concentration levels nor realistic time scales. As a consequence, they cannot be used to explain and predict the outcome of biological experiments that yield quantitative data. However, given a biological network consisting of input (e.g., receptors), intermediate, and output (e.g., transcription factors) signals, they allow studying the input-output relationships through discrete simulation (Samaga, 2013). Boolean models are gaining an increasing interest in reproducing dynamic behaviors, understanding processes, and predicting emerging properties of cellular signaling networks through in-silico experiments. They are emerging as a valid alternative to the quantitative approaches (i.e., based on ordinary differential equations) for exploratory modeling when little is known about reaction kinetics or equilibrium constants in the context of gene expression or signaling. Even though several approaches and software have been recently proposed for logic modeling of biological systems, they are limited to specific contexts and they lack of automation in analyzing biological properties such as complex attractors, and molecule vulnerability. This thesis proposes a platform based on Electronic Design Automation (EDA) technologies for qualitative modeling and simulation of Biological Systems. It aims at overtaking limitations that affect the most recent qualitative tools

Fault-based Analysis of Industrial Cyber-Physical Systems

The fourth industrial revolution called Industry 4.0 tries to bridge the gap between traditional Electronic Design Automation (EDA) technologies and the necessity of innovating in many indus- trial fields, e.g., automotive, avionic, and manufacturing. This complex digitalization process in- volves every industrial facility and comprises the transformation of methodologies, techniques, and tools to improve the efficiency of every industrial process. The enhancement of functional safety in Industry 4.0 applications needs to exploit the studies related to model-based and data-driven anal- yses of the deployed Industrial Cyber-Physical System (ICPS). Modeling an ICPS is possible at different abstraction levels, relying on the physical details included in the model and necessary to describe specific system behaviors. However, it is extremely complicated because an ICPS is com- posed of heterogeneous components related to different physical domains, e.g., digital, electrical, and mechanical. In addition, it is also necessary to consider not only nominal behaviors but even faulty behaviors to perform more specific analyses, e.g., predictive maintenance of specific assets. Nevertheless, these faulty data are usually not present or not available directly from the industrial machinery. To overcome these limitations, constructing a virtual model of an ICPS extended with different classes of faults enables the characterization of faulty behaviors of the system influenced by different faults. In literature, these topics are addressed with non-uniformly approaches and with the absence of standardized and automatic methodologies for describing and simulating faults in the different domains composing an ICPS. This thesis attempts to overcome these state-of-the-art gaps by proposing novel methodologies, techniques, and tools to: model and simulate analog and multi-domain systems; abstract low-level models to higher-level behavioral models; and monitor industrial systems based on the Industrial Internet of Things (IIOT) paradigm. Specifically, the proposed contributions involve the exten- sion of state-of-the-art fault injection practices to improve the ICPSs safety, the development of frameworks for safety operations automatization, and the definition of a monitoring framework for ICPSs. Overall, fault injection in analog and digital models is the state of the practice to en- sure functional safety, as mentioned in the ISO 26262 standard specific for the automotive field. Starting from state-of-the-art defects defined for analog descriptions, new defects are proposed to enhance the IEEE P2427 draft standard for analog defect modeling and coverage. Moreover, dif- ferent techniques to abstract a transistor-level model to a behavioral model are proposed to speed up the simulation of faulty circuits. Therefore, unlike the electrical domain, there is no extensive use of fault injection techniques in the mechanical one. Thus, extending the fault injection to the mechanical and thermal fields allows for supporting the definition and evaluation of more reliable safety mechanisms. Hence, a taxonomy of mechanical faults is derived from the electrical domain by exploiting the physical analogies. Furthermore, specific tools are built for automatically instru- menting different descriptions with multi-domain faults. The entire work is proposed as a basis for supporting the creation of increasingly resilient and secure ICPS that need to preserve functional safety in any operating context

Nuevas técnicas de inyección de fallos en sistemas embebidos mediante el uso de modelos virtuales descritos en el nivel de transacción

Mejor software y más rápido. Este es el desafío que se deriva de la necesidad de construir sistemas cada vez más inteligentes. En cualquier diseño embebido actual, el software es un componente fundamental que dota al sistema de una alta capacidad de configuración, gran número de funcionalidades y elasticidad en el comportamiento del sistema en situaciones excepcionales. Si además el desarrollo del conjunto hardware/software integrado en un System on Chip (SoC), forma parte de un sistema de control crítico donde se deben tener en cuenta requisitos de tolerancia a fallos, la verificación exhaustiva de los mismos consume un porcentaje cada vez más importante de los recursos totales dedicados al desarrollo y puesta en funcionamiento del sistema. En este contexto, el uso de metodologías clásicas de codiseño y coverificación es completamente ineficiente, siendo necesario el uso de nuevas tecnologías y herramientas para el desarrollo y verificación tempranos del software embebido. Entre ellas se puede incluir la propuesta en este trabajo de tesis, la cual aborda el problema mediante el uso de modelos ejecutables del hardware definidos en el nivel de transacción. Debido a los estrictos requisitos de robustez que imperan en el desarrollo de software espacial, es necesario llevar a cabo tareas de verificación en etapas muy tempranas del desarrollo para asegurar que los mecanismos de tolerancia a fallos, avanzados en la especificación del sistema, funcionan adecuadamente. De forma general, es deseable que estas tareas se realicen en paralelo con el desarrollo hardware, anticipando problemas o errores existentes en la especificación del sistema. Además, la verificación completa de los mecanismos de excepción implementados en el software, puede ser imposible de realizar en hardware real ya que los escenarios de fallo deben ser artificial y sistemáticamente generados mediante técnicas de inyección de fallos que permitan realizar campañas de inyección controlables, observables y reproducibles. En esta tesis se describe la investigación, desarrollo y uso de una plataforma virtual denominada "Leon2ViP", con capacidad de inyección de fallos y basada en interfaces SystemC/TLM2 para el desarrollo temprano y verificación de software embebido en el marco del proyecto Solar Orbiter. De esta forma ha sido posible ejecutar y probar exactamente el mismo código binario a ejecutar en el hardware real, pero en un entorno más controlable y determinista. Ello permite la realización de campañas de inyección de fallos muy focalizadas que no serían posible de otra manera. El uso de "\Leon2ViP" ha significado una mejora significante, en términos de coste y tiempo, en el desarrollo y verificación del software de arranque de la unidad de control del instrumento (ICU) del detector de partículas energéticas (EPD) embarcado en Solar Orbiter

Un Framework para la generación automática de ejercicios mediante técnicas de mutación

En este trabajo se describe el diseño e implementación de un entorno que genera ejercicios tipo test de forma automática mediante técnicas de mutación, llamado Wodel-Edu. Wodel-Edu es una extensión de post-procesado para el Lenguaje de Dominio Específico (DSL) Wodel, desarrollado por el grupo MISO, y que proporciona primitivas de alto nivel para mutación de modelos. Para ello, se ha extendido el DSL Wodel con nuevas primitivas de mutación, nuevas estrategias de selección, un registro de las mutaciones aplicadas, un control de la generación de mutantes duplicados, y una comprobación de que los mutantes que se generan son modelos correctos (conformes a su meta-modelo). También se ha dotado a Wodel de un mecanismo extensible que permite registrar distintas acciones de post-procesado sobre los mutantes generados, extensión sobre la que se ha implementado el entorno Wodel-Edu. Wodel-Edu es independiente del dominio, y genera tres formatos diferentes de ejercicios tipo test: el primero, en el que se presentan varios diagramas, y el estudiante ha de decidir cuál es el correcto; el segundo, en el que se presenta un único diagrama, y el estudiante ha de decidir si es correcto, o no; el tercer formato, se presentan una serie de posibles cambios a realizar sobre el diagrama para corregirlo, y el estudiante ha de seleccionar cuáles de estos cambios son correctos. En este trabajo se ha elegido utilizar Wodel-Edu para generar ejercicios de autómatas finitos. Se presenta además una evaluación de la aplicación de ejercicios generada.This work presents the design and development of a framework for the automatic generation of test exercises using mutation techniques, that we call Wodel-Edu. Wodel-Edu is a post-processing extension for the Domain-Specific Language (DSL) Wodel - developed by MISO group - that provides high level primitives for model mutation. We extend the DSL Wodel with new mutation primitives, new selection strategies, a registry of the applied mutations, a duplicated mutant generation control, and a verification that the generated mutants are conforming to the meta-model. We also improve Wodel with an extensible mechanism that allows applying post-processing actions to the generated mutants, and we use this feature to include the Wodel-Edu extension in the Wodel environment. Wodel-Edu is domain independent, and generates three kind of test exercises: the first one, where several diagrams are shown to the student, and he has to choose which one is correct; the second one, where just one diagram is shown to the student, and he has to choose if it is correct or not; and the third kind of exercise, where several changes, that can be applied to the diagram, are presented to the student, and he has to choose which of these changes are correct. In this work, we chose to apply Wodel-Edu to generate finite automata exercises. We also present an evaluation of the generated test application