4,585 research outputs found
Synthesizing Adaptive Test Strategies from Temporal Logic Specifications
Constructing good test cases is difficult and time-consuming, especially if
the system under test is still under development and its exact behavior is not
yet fixed. We propose a new approach to compute test strategies for reactive
systems from a given temporal logic specification using formal methods. The
computed strategies are guaranteed to reveal certain simple faults in every
realization of the specification and for every behavior of the uncontrollable
part of the system's environment. The proposed approach supports different
assumptions on occurrences of faults (ranging from a single transient fault to
a persistent fault) and by default aims at unveiling the weakest one. Based on
well-established hypotheses from fault-based testing, we argue that such tests
are also sensitive for more complex bugs. Since the specification may not
define the system behavior completely, we use reactive synthesis algorithms
with partial information. The computed strategies are adaptive test strategies
that react to behavior at runtime. We work out the underlying theory of
adaptive test strategy synthesis and present experiments for a safety-critical
component of a real-world satellite system. We demonstrate that our approach
can be applied to industrial specifications and that the synthesized test
strategies are capable of detecting bugs that are hard to detect with random
testing
Time-Staging Enhancement of Hybrid System Falsification
Optimization-based falsification employs stochastic optimization algorithms
to search for error input of hybrid systems. In this paper we introduce a
simple idea to enhance falsification, namely time staging, that allows the
time-causal structure of time-dependent signals to be exploited by the
optimizers. Time staging consists of running a falsification solver multiple
times, from one interval to another, incrementally constructing an input signal
candidate. Our experiments show that time staging can dramatically increase
performance in some realistic examples. We also present theoretical results
that suggest the kinds of models and specifications for which time staging is
likely to be effective
Compositional Falsification of Cyber-Physical Systems with Machine Learning Components
Cyber-physical systems (CPS), such as automotive systems, are starting to
include sophisticated machine learning (ML) components. Their correctness,
therefore, depends on properties of the inner ML modules. While learning
algorithms aim to generalize from examples, they are only as good as the
examples provided, and recent efforts have shown that they can produce
inconsistent output under small adversarial perturbations. This raises the
question: can the output from learning components can lead to a failure of the
entire CPS? In this work, we address this question by formulating it as a
problem of falsifying signal temporal logic (STL) specifications for CPS with
ML components. We propose a compositional falsification framework where a
temporal logic falsifier and a machine learning analyzer cooperate with the aim
of finding falsifying executions of the considered model. The efficacy of the
proposed technique is shown on an automatic emergency braking system model with
a perception component based on deep neural networks
Techniques for automated parameter estimation in computational models of probabilistic systems
The main contribution of this dissertation is the design of two new algorithms for automatically synthesizing values of numerical parameters of computational models of complex stochastic systems such that the resultant model meets user-specified behavioral specifications. These algorithms are designed to operate on probabilistic systems – systems that, in general, behave differently under identical conditions. The algorithms work using an approach that combines formal verification and mathematical optimization to explore a model\u27s parameter space. The problem of determining whether a model instantiated with a given set of parameter values satisfies the desired specification is first defined using formal verification terminology, and then reformulated in terms of statistical hypothesis testing. Parameter space exploration involves determining the outcome of the hypothesis testing query for each parameter point and is guided using simulated annealing. The first algorithm uses the sequential probability ratio test (SPRT) to solve the hypothesis testing problems, whereas the second algorithm uses an approach based on Bayesian statistical model checking (BSMC). The SPRT-based parameter synthesis algorithm was used to validate that a given model of glucose-insulin metabolism has the capability of representing diabetic behavior by synthesizing values of three parameters that ensure that the glucose-insulin subsystem spends at least 20 minutes in a diabetic scenario. The BSMC-based algorithm was used to discover the values of parameters in a physiological model of the acute inflammatory response that guarantee a set of desired clinical outcomes. These two applications demonstrate how our algorithms use formal verification, statistical hypothesis testing and mathematical optimization to automatically synthesize parameters of complex probabilistic models in order to meet user-specified behavioral propertie
- …