Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Cyber-physical systems (CPS), such as automotive systems, are starting to include sophisticated machine learning (ML) components. Their correctness, therefore, depends on properties of the inner ML modules. While learning algorithms aim to generalize from examples, they are only as good as the examples provided, and recent efforts have shown that they can produce inconsistent output under small adversarial perturbations. This raises the question: can the output from learning components can lead to a failure of the entire CPS? In this work, we address this question by formulating it as a problem of falsifying signal temporal logic (STL) specifications for CPS with ML components. We propose a compositional falsification framework where a temporal logic falsifier and a machine learning analyzer cooperate with the aim of finding falsifying executions of the considered model. The efficacy of the proposed technique is shown on an automatic emergency braking system model with a perception component based on deep neural networks

Lagrangian Reachabililty

We introduce LRT, a new Lagrangian-based ReachTube computation algorithm that conservatively approximates the set of reachable states of a nonlinear dynamical system. LRT makes use of the Cauchy-Green stretching factor (SF), which is derived from an over-approximation of the gradient of the solution flows. The SF measures the discrepancy between two states propagated by the system solution from two initial states lying in a well-defined region, thereby allowing LRT to compute a reachtube with a ball-overestimate in a metric where the computed enclosure is as tight as possible. To evaluate its performance, we implemented a prototype of LRT in C++/Matlab, and ran it on a set of well-established benchmarks. Our results show that LRT compares very favorably with respect to the CAPD and Flow* tools.Comment: Accepted to CAV 201

Bounded Verification with On-the-Fly Discrepancy Computation

Simulation-based verification algorithms can provide formal safety guarantees for nonlinear and hybrid systems. The previous algorithms rely on user provided model annotations called discrepancy function, which are crucial for computing reachtubes from simulations. In this paper, we eliminate this requirement by presenting an algorithm for computing piece-wise exponential discrepancy functions. The algorithm relies on computing local convergence or divergence rates of trajectories along a simulation using a coarse over-approximation of the reach set and bounding the maximal eigenvalue of the Jacobian over this over-approximation. The resulting discrepancy function preserves the soundness and the relative completeness of the verification algorithm. We also provide a coordinate transformation method to improve the local estimates for the convergence or divergence rates in practical examples. We extend the method to get the input-to-state discrepancy of nonlinear dynamical systems which can be used for compositional analysis. Our experiments show that the approach is effective in terms of running time for several benchmark problems, scales reasonably to larger dimensional systems, and compares favorably with respect to available tools for nonlinear models.Comment: 24 page

Interface-aware signal temporal logic

Safety and security are major concerns in the development of Cyber-Physical Systems (CPS). Signal temporal logic (STL) was proposedas a language to specify and monitor the correctness of CPS relativeto formalized requirements. Incorporating STL into a developmentprocess enables designers to automatically monitor and diagnosetraces, compute robustness estimates based on requirements, andperform requirement falsification, leading to productivity gains inverification and validation activities; however, in its current formSTL is agnostic to the input/output classification of signals, andthis negatively impacts the relevance of the analysis results.In this paper we propose to make the interface explicit in theSTL language by introducing input/output signal declarations. Wethen define new measures of input vacuity and output robustnessthat better reflect the nature of the system and the specification in-tent. The resulting framework, which we call interface-aware signaltemporal logic (IA-STL), aids verification and validation activities.We demonstrate the benefits of IA-STL on several CPS analysisactivities: (1) robustness-driven sensitivity analysis, (2) falsificationand (3) fault localization. We describe an implementation of our en-hancement to STL and associated notions of robustness and vacuityin a prototype extension of Breach, a MATLAB®/Simulink®toolboxfor CPS verification and validation. We explore these methodologi-cal improvements and evaluate our results on two examples fromthe automotive domain: a benchmark powertrain control systemand a hydrogen fuel cell system

L’instruction en famille en Suisse romande : portrait des familles et motivations parentales

L'instruction en famille est une option éducative peu connue, mais en développement. Cette première enquête en Suisse romande documente les raisons de ce choix, les pratiques éducatives et le profil sociodémographique chez 137 familles. Les raisons exprimées par ces parents sont multiples et variées. Leurs principaux facteurs décisionnels sont d’ordre social ou pédagogique: une critique de la motivation et de la socialisation en contexte scolaire, la recherche d’un enrichissement des relations familiales, un projet éducatif poursuivi sous la responsabilité parentale, et une critique des programmes d’études cantonaux et des méthodes pédagogiques de l’école. Les résultats révèlent également des situations de souffrance infantile en milieu scolaire.Home education is a little-known but developing educational option. This first survey in French-speaking Switzerland documents the reasons for this choice, the educational practices and the socio-demographic profile of 137 families. The reasons expressed by these parents are many and varied. Their main decision-making factors are social or pedagogical: a critique of motivation and socialization in the school context, the search for enrichment of family relationships, an educational project pursued under parental responsibility, and a critique of cantonal educational programs and of school teaching methods. The results also reveal situations of childhood suffering in schools.Familienerziehung ist eine wenig bekannte, aber sich entwickelnde Bildungsoption. Diese erste Umfrage in der französischsprachigen Schweiz dokumentiert die Gründe für diese Wahl, die Bildungspraktiken und das soziodemografische Profil von 137 Familien. Die von diesen Eltern geäußerten Gründe sind vielfältig. Ihre Hauptentscheidungsfaktoren sind soziale oder pädagogische: eine Kritik der Motivation und Sozialisation im schulischen Kontext, die Suche nach einer Bereicherung familiärer Beziehungen, ein Bildungsprojekt, das unter elterlicher Verantwortung durchgeführt wird, und eine Kritik der Bildungsprogramme. kantonale Studien und Methoden des Schulunterrichts. Die Ergebnisse zeigen auch Situationen, in denen Kinder in Schulen leiden.L'educazione familiare è un'opzione educativa poco conosciuta ma in via di sviluppo. Questo primo sondaggio nella Svizzera romanda documenta le ragioni di questa scelta, le pratiche educative e il profilo socio-demografico di 137 famiglie. Le ragioni espresse da questi genitori sono molte e varie. I loro principali fattori decisionali sono sociali o pedagogici: una critica della motivazione e della socializzazione nel contesto scolastico, la ricerca di arricchimento delle relazioni familiari, un progetto educativo perseguito sotto la responsabilità genitoriale e una critica dei programmi educativi cantonali e metodi di insegnamento scolastico. I risultati rivelano anche situazioni di sofferenza infantile nelle scuole

Simulation-based reachability analysis for nonlinear systems using componentwise contraction properties

A shortcoming of existing reachability approaches for nonlinear systems is the poor scalability with the number of continuous state variables. To mitigate this problem we present a simulation-based approach where we first sample a number of trajectories of the system and next establish bounds on the convergence or divergence between the samples and neighboring trajectories. We compute these bounds using contraction theory and reduce the conservatism by partitioning the state vector into several components and analyzing contraction properties separately in each direction. Among other benefits this allows us to analyze the effect of constant but uncertain parameters by treating them as state variables and partitioning them into a separate direction. We next present a numerical procedure to search for weighted norms that yield a prescribed contraction rate, which can be incorporated in the reachability algorithm to adjust the weights to minimize the growth of the reachable set

Quantitative Regular Expressions for Arrhythmia Detection Algorithms

Motivated by the problem of verifying the correctness of arrhythmia-detection algorithms, we present a formalization of these algorithms in the language of Quantitative Regular Expressions. QREs are a flexible formal language for specifying complex numerical queries over data streams, with provable runtime and memory consumption guarantees. The medical-device algorithms of interest include peak detection (where a peak in a cardiac signal indicates a heartbeat) and various discriminators, each of which uses a feature of the cardiac signal to distinguish fatal from non-fatal arrhythmias. Expressing these algorithms' desired output in current temporal logics, and implementing them via monitor synthesis, is cumbersome, error-prone, computationally expensive, and sometimes infeasible. In contrast, we show that a range of peak detectors (in both the time and wavelet domains) and various discriminators at the heart of today's arrhythmia-detection devices are easily expressible in QREs. The fact that one formalism (QREs) is used to describe the desired end-to-end operation of an arrhythmia detector opens the way to formal analysis and rigorous testing of these detectors' correctness and performance. Such analysis could alleviate the regulatory burden on device developers when modifying their algorithms. The performance of the peak-detection QREs is demonstrated by running them on real patient data, on which they yield results on par with those provided by a cardiologist.Comment: CMSB 2017: 15th Conference on Computational Methods for Systems Biolog

Robustness Analysis and Behavior Discrimination in Enzymatic Reaction Networks

Characterizing the behavior and robustness of enzymatic networks with numerous variables and unknown parameter values is a major challenge in biology, especially when some enzymes have counter-intuitive properties or switch-like behavior between activation and inhibition. In this paper, we propose new methodological and tool-supported contributions, based on the intuitive formalism of temporal logic, to express in a rigorous manner arbitrarily complex dynamical properties. Our multi-step analysis allows efficient sampling of the parameter space in order to define feasible regions in which the model exhibits imposed or experimentally observed behaviors. In a first step, an algorithmic methodology involving sensitivity analysis is conducted to determine bifurcation thresholds for a limited number of model parameters or initial conditions. In a second step, this boundary detection is supplemented by a global robustness analysis, based on quasi-Monte Carlo approach that takes into account all model parameters. We apply this method to a well-documented enzymatic reaction network describing collagen proteolysis by matrix metalloproteinase MMP2 and membrane type 1 metalloproteinase (MT1-MMP) in the presence of tissue inhibitor of metalloproteinase TIMP2. For this model, our method provides an extended analysis and quantification of network robustness toward paradoxical TIMP2 switching activity between activation or inhibition of MMP2 production. Further implication of our approach is illustrated by demonstrating and analyzing the possible existence of oscillatory behaviors when considering an extended open configuration of the enzymatic network. Notably, we construct bifurcation diagrams that specify key parameters values controlling the co-existence of stable steady and non-steady oscillatory proteolytic dynamics

Towards Physical Hybrid Systems

Some hybrid systems models are unsafe for mathematically correct but physically unrealistic reasons. For example, mathematical models can classify a system as being unsafe on a set that is too small to have physical importance. In particular, differences in measure zero sets in models of cyber-physical systems (CPS) have significant mathematical impact on the mathematical safety of these models even though differences on measure zero sets have no tangible physical effect in a real system. We develop the concept of "physical hybrid systems" (PHS) to help reunite mathematical models with physical reality. We modify a hybrid systems logic (differential temporal dynamic logic) by adding a first-class operator to elide distinctions on measure zero sets of time within CPS models. This approach facilitates modeling since it admits the verification of a wider class of models, including some physically realistic models that would otherwise be classified as mathematically unsafe. We also develop a proof calculus to help with the verification of PHS.Comment: CADE 201