Addressing Complexity and Intelligence in Systems Dependability Evaluation

Engineering and computing systems are increasingly complex, intelligent, and open adaptive. When it comes to the dependability evaluation of such systems, there are certain challenges posed by the characteristics of “complexity” and “intelligence”. The first aspect of complexity is the dependability modelling of large systems with many interconnected components and dynamic behaviours such as Priority, Sequencing and Repairs. To address this, the thesis proposes a novel hierarchical solution to dynamic fault tree analysis using Semi-Markov Processes. A second aspect of complexity is the environmental conditions that may impact dependability and their modelling. For instance, weather and logistics can influence maintenance actions and hence dependability of an offshore wind farm. The thesis proposes a semi-Markov-based maintenance model called “Butterfly Maintenance Model (BMM)” to model this complexity and accommodate it in dependability evaluation. A third aspect of complexity is the open nature of system of systems like swarms of drones which makes complete design-time dependability analysis infeasible. To address this aspect, the thesis proposes a dynamic dependability evaluation method using Fault Trees and Markov-Models at runtime.The challenge of “intelligence” arises because Machine Learning (ML) components do not exhibit programmed behaviour; their behaviour is learned from data. However, in traditional dependability analysis, systems are assumed to be programmed or designed. When a system has learned from data, then a distributional shift of operational data from training data may cause ML to behave incorrectly, e.g., misclassify objects. To address this, a new approach called SafeML is developed that uses statistical distance measures for monitoring the performance of ML against such distributional shifts. The thesis develops the proposed models, and evaluates them on case studies, highlighting improvements to the state-of-the-art, limitations and future work

Putting Teeth into Open Architectures: Infrastructure for Reducing the Need for Retesting

Proceedings Paper (for Acquisition Research Program)The Navy is currently implementing the open-architecture framework for developing joint interoperable systems that adapt and exploit open-system design principles and architectures. This raises concerns about how to practically achieve dependability in software-intensive systems with many possible configurations when: 1) the actual configuration of the system is subject to frequent and possibly rapid change, and 2) the environment of typical reusable subsystems is variable and unpredictable. Our preliminary investigations indicate that current methods for achieving dependability in open architectures are insufficient. Conventional methods for testing are suited for stovepipe systems and depend strongly on the assumptions that the environment of a typical system is fixed and known in detail to the quality-assurance team at test and evaluation time. This paper outlines new approaches to quality assurance and testing that are better suited for providing affordable reliability in open architectures, and explains some of the additional technical features that an Open Architecture must have in order to become a Dependable Open Architecture.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited

System-of-Systems Complexity

The global availability of communication services makes it possible to interconnect independently developed systems, called constituent systems, to provide new synergistic services and more efficient economic processes. The characteristics of these new Systems-of-Systems are qualitatively different from the classic monolithic systems. In the first part of this presentation we elaborate on these differences, particularly with respect to the autonomy of the constituent systems, to dependability, continuous evolution, and emergence. In the second part we look at a SoS from the point of view of cognitive complexity. Cognitive complexity is seen as a relation between a model of an SoS and the observer. In order to understand the behavior of a large SoS we have to generate models of adequate simplicity, i.e, of a cognitive complexity that can be handled by the limited capabilities of the human mind. We will discuss the importance of properly specifying and placing the relied-upon message interfaces between the constituent systems that form an open SoS and discuss simplification strategies that help to reduce the cognitive complexity.Comment: In Proceedings AiSoS 2013, arXiv:1311.319

Ensuring Cyber-Security in Smart Railway Surveillance with SHIELD

Modern railways feature increasingly complex embedded computing systems for surveillance, that are moving towards fully wireless smart-sensors. Those systems are aimed at monitoring system status from a physical-security viewpoint, in order to detect intrusions and other environmental anomalies. However, the same systems used for physical-security surveillance are vulnerable to cyber-security threats, since they feature distributed hardware and software architectures often interconnected by ‘open networks’, like wireless channels and the Internet. In this paper, we show how the integrated approach to Security, Privacy and Dependability (SPD) in embedded systems provided by the SHIELD framework (developed within the EU funded pSHIELD and nSHIELD research projects) can be applied to railway surveillance systems in order to measure and improve their SPD level. SHIELD implements a layered architecture (node, network, middleware and overlay) and orchestrates SPD mechanisms based on ontology models, appropriate metrics and composability. The results of prototypical application to a real-world demonstrator show the effectiveness of SHIELD and justify its practical applicability in industrial settings

Assurance of open systems dependability: developing a framework for automotive security and safety

We describe how a security informed analysis of the open systems dependability model of DEOS can be used to frame the problem of open systems and security. Together with an approach for analysing industry objectives based on claims, arguments and evidence (CAE), we develop a set of principles and rationale for the security and safety of road transport systems. The associated CAE will provide a generic template for a security informed safety case and supports standardization activities for security-informed safety

Analysing security properties using refinement

Security properties are essential in open and distributed environments with high dependability requirements. An approach to development and analysis of safety- and security-critical systems based on refinement as the central concept can offer an integrated solution. We analyse the Online Certificate Status Protocol (OCSP), showing how to use refinement as an interference analysis tool for secure communication protocols and intruders

Verl\"assliche Software im 21. Jahrhundert

Software is the main innovation driver in many different areas, like cloud services, autonomous driving, connected medical devices, and high-frequency trading. All these areas have in common that they require high dependability. In this paper, we discuss challenges and research directions imposed by these new areas on guaranteeing the dependability. On the one hand challenges include characteristics of the systems themselves, e. g., open systems and ad-hoc structures. On the other hand, we see new aspects of dependability like behavioral traceability.Comment: 6 pages, in German, 1 figur

An overview of fault tree analysis and its application in model based dependability analysis

YesFault Tree Analysis (FTA) is a well-established and well-understood technique, widely used for dependability evaluation of a wide range of systems. Although many extensions of fault trees have been proposed, they suffer from a variety of shortcomings. In particular, even where software tool support exists, these analyses require a lot of manual effort. Over the past two decades, research has focused on simplifying dependability analysis by looking at how we can synthesise dependability information from system models automatically. This has led to the field of model-based dependability analysis (MBDA). Different tools and techniques have been developed as part of MBDA to automate the generation of dependability analysis artefacts such as fault trees. Firstly, this paper reviews the standard fault tree with its limitations. Secondly, different extensions of standard fault trees are reviewed. Thirdly, this paper reviews a number of prominent MBDA techniques where fault trees are used as a means for system dependability analysis and provides an insight into their working mechanism, applicability, strengths and challenges. Finally, the future outlook for MBDA is outlined, which includes the prospect of developing expert and intelligent systems for dependability analysis of complex open systems under the conditions of uncertainty