Blaming in component-based real-time systems

International audienceIn component-based safety-critical real-time systems it is crucial to determine which com-ponent(s) caused the violation of a required system-level safety property, be it to issue a precise alert, or to determine liability of component providers. In this paper we present an approach for blaming in real-time systems whose component specifications are given as timed automata. The analysis is based on a single execution trace violating a safety property P. We formalize blaming using counterfactual reasoning ("what would have been the outcome if component C had behaved correctly?") to distinguish component failures that actually con-tributed to the outcome from failures that had no impact on the violation of P. We then show how to effectively implement blaming by reducing it to a model-checking problem for timed automata, and demonstrate the feasibility of our approach on the models of a pacemaker and of a chemical reactor

Counterfactual Causality from First Principles?

In this position paper we discuss three main shortcomings of existing approaches to counterfactual causality from the computer science perspective, and sketch lines of work to try and overcome these issues: (1) causality definitions should be driven by a set of precisely specified requirements rather than specific examples; (2) causality frameworks should support system dynamics; (3) causality analysis should have a well-understood behavior in presence of abstraction.Comment: In Proceedings CREST 2017, arXiv:1710.0277

A Component-oriented Framework for Autonomous Agents

The design of a complex system warrants a compositional methodology, i.e., composing simple components to obtain a larger system that exhibits their collective behavior in a meaningful way. We propose an automaton-based paradigm for compositional design of such systems where an action is accompanied by one or more preferences. At run-time, these preferences provide a natural fallback mechanism for the component, while at design-time they can be used to reason about the behavior of the component in an uncertain physical world. Using structures that tell us how to compose preferences and actions, we can compose formal representations of individual components or agents to obtain a representation of the composed system. We extend Linear Temporal Logic with two unary connectives that reflect the compositional structure of the actions, and show how it can be used to diagnose undesired behavior by tracing the falsification of a specification back to one or more culpable components

A Hybrid Approach to Causality Analysis

In component-based safety-critical systems, when a system safety property is violated, it is necessary to analyze which components are the cause. Given a system execution trace that exhibits component faults leading to a property violation, our causality analysis formalizes a notion of counterfactual reasoning (\what would the system behavior be if a component had been correct? ) and algorithmically derives such alternative system behaviors, without re-executing the system itself. In this paper, we show that we can improve precision of the analysis if 1) we can emulate execution of components instead of relying on their contracts, and 2) take into consideration input/output dependencies between components to avoid blaming components for faults induced by other components. We demonstrate the utility of the extended analysis with a case study for a closed-loop patient-controlled analgesia system

Fault Ascription in Concurrent Systems

Fault diagnosis is becoming increasingly important and difficult with the growing pervasiveness and complexity of computer systems. We propose in this paper a general semantic framework for fault ascription, a precise form of fault diagnosis that relies on counterfactual analysis for identifying necessary and sufficient causes of faults in component-based systems. Our framework relies on configuration structures to handle concurrent systems, partial and distributed observations in a uniform way. It defines basic conditions for a counterfactual analysis of necessary and sufficient causes, and it presents a refined analysis that conforms to our basic conditions while avoiding various infelicities

Program Actions as Actual Causes: A Building Block for Accountability

Abstract-Protocols for tasks such as authentication, electronic voting, and secure multiparty computation ensure desirable security properties if agents follow their prescribed programs. However, if some agents deviate from their prescribed programs and a security property is violated, it is important to hold agents accountable by determining which deviations actually caused the violation. Motivated by these applications, we initiate a formal study of program actions as actual causes. Specifically, we define in an interacting program model what it means for a set of program actions to be an actual cause of a violation. We present a sound technique for establishing program actions as actual causes. We demonstrate the value of this formalism in two ways. First, we prove that violations of a specific class of safety properties always have an actual cause. Thus, our definition applies to relevant security properties. Second, we provide a cause analysis of a representative protocol designed to address weaknesses in the current public key certification infrastructure

Causality Analysis and Fault Ascription in Component-Based Systems

This article introduces a general framework for fault ascription, which consists in identifying, within a multi-component system, the components whose faulty behavior has caused the failure of said system. Our framework uses configuration structures as a general semantical model to handle truly concurrent executions, partial and distributed observations in a uniform way. We define a set of expected properties for counterfactual analysis, and present a refined analysis that conforms to our requirements. This contrasts with current practice of evaluating definitions of counterfactual causality a posteriori on a set of toy examples. As an early study of the behavior of our analysis under abstraction we establish its monotony under refinement.Cet article introduit un cadre général pour l’attribution de fautes qui consiste à identifier, dans un système à composants, les composants dont le comportement incorrect a causé le dysfonctionnement du système. Nous définissons un ensemble de propriétés attendues de l’analyse contrefactuelle, et nous présentons une analyse raffinée qui satisfait ces besoins. Ceci contraste avec la pratique courante d’évaluer les définitions de causalité contrefactuelle a posteriori sur un ensemble d’exemples jouets. Nous établissons la monotonie de notre analyse sous différentes notions de raffinement

Formal Techniques for Component-based Design of Embedded Systems

Embedded systems have become ubiquitous - from avionics and automotive over consumer electronics to medical devices. Failures may entailmaterial damage or compromise safety of human beings. At the same time, shorter product cycles, together with fast growing complexity of the systems to be designed, create a tremendous need for rigorous design techniques. The goal of component-based construction is to build complex systems from simpler components that are well understood and can be (re)used so as to accelerate the design process. This document presents a summary of the formal techniques for component-based design of embedded systems I have (co-)developed