Re-use of tests and arguments for assesing dependable mixed-critically systems

The safety assessment of mixed-criticality systems (MCS) is a challenging activity due to system heterogeneity, design constraints and increasing complexity. The foundation for MCSs is the integrated architecture paradigm, where a compact hardware comprises multiple execution platforms and communication interfaces to implement concurrent functions with different safety requirements. Besides a computing platform providing adequate isolation and fault tolerance mechanism, the development of an MCS application shall also comply with the guidelines defined by the safety standards. A way to lower the overall MCS certification cost is to adopt a platform-based design (PBD) development approach. PBD is a model-based development (MBD) approach, where separate models of logic, hardware and deployment support the analysis of the resulting system properties and behaviour. The PBD development of MCSs benefits from a composition of modular safety properties (e.g. modular safety cases), which support the derivation of mixed-criticality product lines. The validation and verification (V&V) activities claim a substantial effort during the development of programmable electronics for safety-critical applications. As for the MCS dependability assessment, the purpose of the V&V is to provide evidences supporting the safety claims. The model-based development of MCSs adds more V&V tasks, because additional analysis (e.g., simulations) need to be carried out during the design phase. During the MCS integration phase, typically hardware-in-the-loop (HiL) plant simulators support the V&V campaigns, where test automation and fault-injection are the key to test repeatability and thorough exercise of the safety mechanisms. This dissertation proposes several V&V artefacts re-use strategies to perform an early verification at system level for a distributed MCS, artefacts that later would be reused up to the final stages in the development process: a test code re-use to verify the fault-tolerance mechanisms on a functional model of the system combined with a non-intrusive software fault-injection, a model to X-in-the-loop (XiL) and code-to-XiL re-use to provide models of the plant and distributed embedded nodes suited to the HiL simulator, and finally, an argumentation framework to support the automated composition and staged completion of modular safety-cases for dependability assessment, in the context of the platform-based development of mixed-criticality systems relying on the DREAMS harmonized platform.La dificultad para evaluar la seguridad de los sistemas de criticidad mixta (SCM) aumenta con la heterogeneidad del sistema, las restricciones de diseño y una complejidad creciente. Los SCM adoptan el paradigma de arquitectura integrada, donde un hardware embebido compacto comprende múltiples plataformas de ejecución e interfaces de comunicación para implementar funciones concurrentes y con diferentes requisitos de seguridad. Además de una plataforma de computación que provea un aislamiento y mecanismos de tolerancia a fallos adecuados, el desarrollo de una aplicación SCM además debe cumplir con las directrices definidas por las normas de seguridad. Una forma de reducir el coste global de la certificación de un SCM es adoptar un enfoque de desarrollo basado en plataforma (DBP). DBP es un enfoque de desarrollo basado en modelos (DBM), en el que modelos separados de lógica, hardware y despliegue soportan el análisis de las propiedades y el comportamiento emergente del sistema diseñado. El desarrollo DBP de SCMs se beneficia de una composición modular de propiedades de seguridad (por ejemplo, casos de seguridad modulares), que facilitan la definición de líneas de productos de criticidad mixta. Las actividades de verificación y validación (V&V) representan un esfuerzo sustancial durante el desarrollo de aplicaciones basadas en electrónica confiable. En la evaluación de la seguridad de un SCM el propósito de las actividades de V&V es obtener las evidencias que apoyen las aseveraciones de seguridad. El desarrollo basado en modelos de un SCM incrementa las tareas de V&V, porque permite realizar análisis adicionales (por ejemplo, simulaciones) durante la fase de diseño. En las campañas de pruebas de integración de un SCM habitualmente se emplean simuladores de planta hardware-in-the-loop (HiL), en donde la automatización de pruebas y la inyección de faltas son la clave para la repetitividad de las pruebas y para ejercitar completamente los mecanismos de tolerancia a fallos. Esta tesis propone diversas estrategias de reutilización de artefactos de V&V para la verificación temprana de un MCS distribuido, artefactos que se emplearán en ulteriores fases del desarrollo: la reutilización de código de prueba para verificar los mecanismos de tolerancia a fallos sobre un modelo funcional del sistema combinado con una inyección de fallos de software no intrusiva, la reutilización de modelo a X-in-the-loop (XiL) y código a XiL para obtener modelos de planta y nodos distribuidos aptos para el simulador HiL y, finalmente, un marco de argumentación para la composición automatizada y la compleción escalonada de casos de seguridad modulares, en el contexto del desarrollo basado en plataformas de sistemas de criticidad mixta empleando la plataforma armonizada DREAMS.Kritikotasun nahastuko sistemen segurtasun ebaluazioa jarduera neketsua da beraien heterogeneotasuna dela eta. Sistema hauen oinarria arkitektura integratuen paradigman datza, non hardware konpaktu batek exekuzio plataforma eta komunikazio interfaze ugari integratu ahal dituen segurtasun baldintza desberdineko funtzio konkurrenteak inplementatzeko. Konputazio plataformek isolamendu eta akatsen aurkako mekanismo egokiak emateaz gain, segurtasun arauek definituriko jarraibideak jarraitu behar dituzte kritikotasun mistodun aplikazioen garapenean. Sistema hauen zertifikazio prozesuaren kostua murrizteko aukera bat plataformetan oinarritutako garapenean (PBD) datza. Garapen planteamendu hau modeloetan oinarrituriko garapena da (MBD) non modeloaren logika, hardware eta garapen desberdinak sistemaren propietateen eta portaeraren aurka aztertzen diren. Kritikotasun mistodun sistemen PBD garapenak etekina ateratzen dio moduluetan oinarrituriko segurtasun propietateei, adibidez: segurtasun kasu modularrak (MSC). Modulu hauek kritikotasun mistodun produktu-lerroak ere hartzen dituzte kontutan. Berifikazio eta balioztatze (V&V) jarduerek esfortzu kontsideragarria eskatzen dute segurtasun-kiritikoetarako elektronika programagarrien garapenean. Kritikotasun mistodun sistemen konfiantzaren ebaluazioaren eta V&V jardueren helburua segurtasun eskariak jasotzen dituzten frogak proportzionatzea da. Kritikotasun mistodun sistemen modelo bidezko garapenek zeregin gehigarriak atxikitzen dizkio V&V jarduerari, fase honetan analisi gehigarriak (hots, simulazioak) zehazten direlako. Bestalde, kritikotasun mistodun sistemen integrazio fasean, hardware-in-the-loop (Hil) simulazio plantek V&V iniziatibak sostengatzen dituzte non testen automatizazioan eta akatsen txertaketan funtsezko jarduerak diren. Jarduera hauek frogen errepikapena eta segurtasun mekanismoak egiaztzea ahalbidetzen dute. Tesi honek V&V artefaktuen berrerabilpenerako estrategiak proposatzen ditu, kritikotasun mistodun sistemen egiaztatze azkarrerako sistema mailan eta garapen prozesuko azken faseetaraino erabili daitezkeenak. Esate baterako, test kodearen berrabilpena akats aurkako mekanismoak egiaztatzeko, modelotik X-in-the-loop (XiL)-ra eta kodetik XiL-rako konbertsioa HiL simulaziorako eta argumentazio egitura bat DREAMS Europear proiektuan definituriko arkitektura estiloan oinarrituriko segurtasun kasu modularrak automatikoki eta gradualki sortzeko

Dynamic Partial Reconfiguration for Dependable Systems

Moore’s law has served as goal and motivation for consumer electronics manufacturers in the last decades. The results in terms of processing power increase in the consumer electronics devices have been mainly achieved due to cost reduction and technology shrinking. However, reducing physical geometries mainly affects the electronic devices’ dependability, making them more sensitive to soft-errors like Single Event Transient (SET) of Single Event Upset (SEU) and hard (permanent) faults, e.g. due to aging effects. Accordingly, safety critical systems often rely on the adoption of old technology nodes, even if they introduce longer design time w.r.t. consumer electronics. In fact, functional safety requirements are increasingly pushing industry in developing innovative methodologies to design high-dependable systems with the required diagnostic coverage. On the other hand commercial off-the-shelf (COTS) devices adoption began to be considered for safety-related systems due to real-time requirements, the need for the implementation of computationally hungry algorithms and lower design costs. In this field FPGA market share is constantly increased, thanks to their flexibility and low non-recurrent engineering costs, making them suitable for a set of safety critical applications with low production volumes. The works presented in this thesis tries to face new dependability issues in modern reconfigurable systems, exploiting their special features to take proper counteractions with low impacton performances, namely Dynamic Partial Reconfiguration

Mixed-Criticality Systems on Commercial-Off-the-Shelf Multi-Processor Systems-on-Chip

Avionics and space industries are struggling with the adoption of technologies like multi-processor system-on-chips (MPSoCs) due to strict safety requirements. This thesis propose a new reference architecture for MPSoC-based mixed-criticality systems (MCS) - i.e., systems integrating applications with different level of criticality - which are a common use case for aforementioned industries. This thesis proposes a system architecture capable of granting partitioning - which is, for short, the property of fault containment. It is based on the detection of spatial and temporal interference, and has been named the online detection of interference (ODIn) architecture. Spatial partitioning requires that an application is not able to corrupt resources used by a different application. In the architecture proposed in this thesis, spatial partitioning is implemented using type-1 hypervisors, which allow definition of resource partitions. An application running in a partition can only access resources granted to that partition, therefore it cannot corrupt resources used by applications running in other partitions. Temporal partitioning requires that an application is not able to unexpectedly change the execution time of other applications. In the proposed architecture, temporal partitioning has been solved using a bounded interference approach, composed of an offline analysis phase and an online safety net. The offline phase is based on a statistical profiling of a metric sensitive to temporal interference’s, performed in nominal conditions, which allows definition of a set of three thresholds: 1. the detection threshold TD; 2. the warning threshold TW ; 3. the α threshold. Two rules of detection are defined using such thresholds: Alarm rule When the value of the metric is above TD. Warning rule When the value of the metric is in the warning region [TW ;TD] for more than α consecutive times. ODIn’s online safety-net exploits performance counters, available in many MPSoC architectures; such counters are configured at bootstrap to monitor the selected metric(s), and to raise an interrupt request (IRQ) in case the metric value goes above TD, implementing the alarm rule. The warning rule is implemented in a software detection module, which reads the value of performance counters when the monitored task yields control to the scheduler and reset them if there is no detection. ODIn also uses two additional detection mechanisms: 1. a control flow check technique, based on compile-time defined block signatures, is implemented through a set of watchdog processors, each monitoring one partition. 2. a timeout is implemented through a system watchdog timer (SWDT), which is able to send an external signal when the timeout is violated. The recovery actions implemented in ODIn are: • graceful degradation, to react to IRQs of WDPs monitoring non-critical applications or to warning rule violations; it temporarily stops non-critical applications to grant resources to the critical application; • hard recovery, to react to the SWDT, to the WDP of the critical application, or to alarm rule violations; it causes a switch to a hot stand-by spare computer. Experimental validation of ODIn was performed on two hardware platforms: the ZedBoard - dual-core - and the Inventami board - quad-core. A space benchmark and an avionic benchmark were implemented on both platforms, composed by different modules as showed in Table 1 Each version of the final application was evaluated through fault injection (FI) campaigns, performed using a specifically designed FI system. There were three types of FI campaigns: 1. HW FI, to emulate single event effects; 2. SW FI, to emulate bugs in non-critical applications; 3. artificial bug FI, to emulate a bug in non-critical applications introducing unexpected interference on the critical application. Experimental results show that ODIn is resilient to all considered types of faul

Experimental analysis of computer system dependability

This paper reviews an area which has evolved over the past 15 years: experimental analysis of computer system dependability. Methodologies and advances are discussed for three basic approaches used in the area: simulated fault injection, physical fault injection, and measurement-based analysis. The three approaches are suited, respectively, to dependability evaluation in the three phases of a system's life: design phase, prototype phase, and operational phase. Before the discussion of these phases, several statistical techniques used in the area are introduced. For each phase, a classification of research methods or study topics is outlined, followed by discussion of these methods or topics as well as representative studies. The statistical techniques introduced include the estimation of parameters and confidence intervals, probability distribution characterization, and several multivariate analysis methods. Importance sampling, a statistical technique used to accelerate Monte Carlo simulation, is also introduced. The discussion of simulated fault injection covers electrical-level, logic-level, and function-level fault injection methods as well as representative simulation environments such as FOCUS and DEPEND. The discussion of physical fault injection covers hardware, software, and radiation fault injection methods as well as several software and hybrid tools including FIAT, FERARI, HYBRID, and FINE. The discussion of measurement-based analysis covers measurement and data processing techniques, basic error characterization, dependency analysis, Markov reward modeling, software-dependability, and fault diagnosis. The discussion involves several important issues studies in the area, including fault models, fast simulation techniques, workload/failure dependency, correlated failures, and software fault tolerance

Application of Advanced Early Warning Systems with Adaptive Protection

This project developed and field-tested two methods of Adaptive Protection systems utilizing synchrophasor data. One method detects conditions of system stress that can lead to unintended relay operation, and initiates a supervisory signal to modify relay response in real time to avoid false trips. The second method detects the possibility of false trips of impedance relays as stable system swings “encroach” on the relays’ impedance zones, and produces an early warning so that relay engineers can re-evaluate relay settings. In addition, real-time synchrophasor data produced by this project was used to develop advanced visualization techniques for display of synchrophasor data to utility operators and engineers

Käyttövarmuus liikkuvassa elektroniikassa

Requirement for highly dependable machinery control system is growing from increased complexity of control systems and their ability to control critical machinery functions. This has been noticed by legal authorities and governing legislation is becoming effective. Legal requirements can be met by using methodology based on adequate functional safety standards. Standards require certain tools and methods for product life cycle planning and implementation. Development and operational work flow shall be adapted to fulfill those requirements. Main focus in the study is to interpret standard requirements to process changes and to understand basic philosophy for reliable programmable system hardware. Standards IEC 61508 and ISO 25119 are referenced as main source for requirements. Dependability is based on failure avoidance and control. Study introduces several failure avoidance tools and methods. V-model based work flow is adapted to industry specific requirements. Model includes life cycle approach, deliverable list and assessment checklist for safety related project flow. Documentation structure for good traceability is introduced for unit specification. System level analysis is based on failure mode and effect studies and usage of fault tree modeling helps to understand links between events. Safety level targeting model based on risk graph is introduced for usage in machinery-control-systems. Usage of tools and methods was tested in machinerycontrol-system concept development. Developed concept is intended for operator interface and control tasks. Tools proved to be usable for engineering project and fully implemented documentation model shall fulfill basic assessment requirements. Developed concept itself is usable in critical control systems, but some fine tuning is needed. /Kir11Vaatimukset elektroniikan käyttövarmuudelle ovat kasvaneet viimeksi kuluneiden kymmenen vuoden aikana voimakkaasti, koska ohjausjärjestelmistä on tullut monimutkaisia ja ne ovat korvanneet mekaanisia turvalaitteita. Erityisesti turvallisuuskriittisten toimintojen ohjaaminen on yleistynyt. Muutos on huomattu myös tuoteturvallisuutta valvovien viranomaisten toimesta ja ohjausjärjestelmien toimintaa ja suunnittelua koskevia vaatimuksia on kehitetty. Lainsäädännölliset vaatimukset ovat tulossa voimaan lähiaikoina eri laiteympäristöille ja ne perustuvat olemassa oleviin toiminnallista turvallisuutta koskeviin standardeihin. Standardien mukaisuus savutetaan käyttämällä niissä kuvattuja toimintatapoja ja työkaluja koko tuotteen elinkaaren aikana aina esisuunnittelusta käytöstäpoistoon asti. Suunnittelu, kokoonpano ja asennustyön kulku tulee sovittaa täyttämään nuo vaatimukset. Työn tarkoituksena on selvittää keskeiset toimintatavat ja työkalut liittyen standardeihin ja käyttövarman elektroniikan toimintoihin. Toimintatapoja, työkaluja ja teknisiäratkaisuja arvioidaan myös työn aikana suunnitellun koneenohjauskonseptin kautta. Käyttövarma koneenohajausjärjestelmä mahdollistaa laitteen luotettavan käytön vaarantamatta ihmisiä tai ympäristöä. Tärkeä osa käyttövarmuutta on myös käytön jatkuvuus ja huollettavuus suunnitellusti. Käyttövarmuus tuleekin nähdä tuotteeseen sisäänrakennettuna ominaisuutena suunnittelun, valmistuksen ja käytön aikana. Käyttövarmuus perustuu virhetilanteiden välttämiseen ja niiden vaikutusten kontrollointiin. Virheet voidaan jakaa kahteen päälohkoon. Satunnaisia virheitä esiintyy laitteen eliniän aikana, mutta niiden aiheuttamia vaikutuksia tulee kontrolloida ja pienentää suunnitellusti. Systemaattisia virheitä esiintyy järjestelmässä moninaisista syistä johtuen. Vaarallista vikaantumista voidaan välttää neljällä perustavalla. Laitteiston tulee vikaantua ennustettavalla tavalla. Laitteisto arkkitehtuurin valinnalla voidaan välttää turvallisuuden kannalta kriittisten pullonkaulojen muodostumista. Oikeita toimintapoja nuodattamalla voidaan vähentää systemaattisia virheitä laitteiston toteutuksessa. Standardit edellyttävät V-mallin mukaista toimintamallia tuotteen vaatimustenmukaisuuden varmistamiseksi. V-mallin rakenteen mukainen dokumentaatio tarjoaa jäljitettävyyden vaatimusten ja testauksen varmentamiseen. Koneenohjauksessa tyypillinen konsepti on hajautettu järjestelmä, jossa usein käyttöliittymä ja varsinainen ohjaus on jaettu eri yksiköihin. Käyttöliittymäyksikkö on sijoitettu lähelle käyttäjää ja varsinainen ohjausyksikkö on kytketty käyttöliittymälaitteeseen sarjaliikenneliitynnällä. Turvallisuuden varmistamiseksi kommunikaatio on kahdennettu ja käytetty sarjaliikenneprotokolla noudattaa CAN standardia. CAN standardi tarjoaa itsessään hyvin virheensietokykyisen kommunikaation ja kahdennus varmistaa toiminnan fyysisten virheiden varalle. Käyttöliittymän tehtävänä on varmistaa oikeiden käskyjen välittäminen oikea-aikaisesti muulle ohjausjärjestelmälle käyttäjän niin halutessa. Ohjausyksiköt valvovat järjestelmän tilaa ja toimintaympäristöä ja tekevät päätöksen komennon toteuttamisesta turvallisuuden sallimissa rajoissa. Järjestelmää analysoitiin ja määriteltiin turvallisuuden vaatimat eheystasot laitteiston toteutukselle. Tyypillisessä koneenohjausjärjestelmässä riskit liittyvät usein käyttäjän vaarantumiseen. Käyttäjää ja yksittäisiä sivullisia vaarantavan vikaantumiset vaativat jonkin verran keskimääräistä tasoa korkeampia turvallisuus eheystasoja. Tyypillisesti vaatimus on eheystaso kaksi. Konseptin vaatimustenmukaisuuden suunnitteluun ja varmistamiseen käytettiin lohkokaaviotasolla vikapuu-, vikamuoto- ja vaikutusanalyysejä. Komponenttitason vikamuoto-, vaikutus- ja kriittisyysanalyysillä (FMEDA) varmistettiin suunnitellun toteutuksen vaatimusten täyttymistä. Konseptin sinänsä havaittiin täyttävän perusvaatimukset hyvin, mutta tiettyjä yksityiskohtia erityisesti yksiköiden yhteisissä osissa tulee parantaa. Järjestelmäsuunnitteluvaiheessa tulee käyttövarmuus asioita miettiä kokonaisuutena. Eri käyttövarmuus näkökohtien välillä syntyy ristiriitaisuuksia ja sovelluksen kannalta oikea katsantokanta kannattaa valita. Usein lainmukaisuus tulee varmistaa ja sen jälkeen kohdistaa suuntaus asiakasvaatimusten mukaan. Siirtyminen hallittuun turvakriittiseen järjestelmäsuunnitteluun on usein asennekysymys organisaatiotasolla ja teknisesti ottaen ratkaisut kannattaa pitää mahdollisimman yksinkertaisina

A fault-tolerant multiprocessor architecture for aircraft, volume 1

A fault-tolerant multiprocessor architecture is reported. This architecture, together with a comprehensive information system architecture, has important potential for future aircraft applications. A preliminary definition and assessment of a suitable multiprocessor architecture for such applications is developed