25 research outputs found
Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics
Automated synthesis of provably correct controllers for cyber-physical
systems is crucial for deployment in safety-critical scenarios. However, hybrid
features and stochastic or unknown behaviours make this problem challenging. We
propose a method for synthesising controllers for Markov jump linear systems
(MJLSs), a class of discrete-time models for cyber-physical systems, so that
they certifiably satisfy probabilistic computation tree logic (PCTL) formulae.
An MJLS consists of a finite set of stochastic linear dynamics and discrete
jumps between these dynamics that are governed by a Markov decision process
(MDP). We consider the cases where the transition probabilities of this MDP are
either known up to an interval or completely unknown. Our approach is based on
a finite-state abstraction that captures both the discrete (mode-jumping) and
continuous (stochastic linear) behaviour of the MJLS. We formalise this
abstraction as an interval MDP (iMDP) for which we compute intervals of
transition probabilities using sampling techniques from the so-called 'scenario
approach', resulting in a probabilistically sound approximation. We apply our
method to multiple realistic benchmark problems, in particular, a temperature
control and an aerial vehicle delivery problem.Comment: 14 pages, 6 figures, under review at QES
Demand flexibility management for buildings-to-grid integration with uncertain generation
Contains fulltext :
228323.pdf (publisher's version ) (Open Access
Correct-by-construction reach-avoid control of partially observable linear stochastic systems
We study feedback controller synthesis for reach-avoid control of discrete-time, linear time-invariant (LTI) systems with Gaussian process and measurement noise. The problem is to compute a controller such that, with at least some required probability, the system reaches a desired goal state in finite time while avoiding unsafe states. Due to stochasticity and nonconvexity, this problem does not admit exact algorithmic or closed-form solutions in general. Our key contribution is a correct-by-construction controller synthesis scheme based on a finite-state abstraction of a Gaussian belief over the unmeasured state, obtained using a Kalman filter. We formalize this abstraction as a Markov decision process (MDP). To be robust against numerical imprecision in approximating transition probabilities, we use MDPs with intervals of transition probabilities. By construction, any policy on the abstraction can be refined into a piecewise linear feedback controller for the LTI system. We prove that the closed-loop LTI system under this controller satisfies the reach-avoid problem with at least the required probability. The numerical experiments show that our method is able to solve reach-avoid problems for systems with up to 6D state spaces, and with control input constraints that cannot be handled by methods such as the rapidly-exploring random belief trees (RRBT)
Decision-Making Under Uncertainty: Beyond Probabilities
This position paper reflects on the state-of-the-art in decision-making under
uncertainty. A classical assumption is that probabilities can sufficiently
capture all uncertainty in a system. In this paper, the focus is on the
uncertainty that goes beyond this classical interpretation, particularly by
employing a clear distinction between aleatoric and epistemic uncertainty. The
paper features an overview of Markov decision processes (MDPs) and extensions
to account for partial observability and adversarial behavior. These models
sufficiently capture aleatoric uncertainty but fail to account for epistemic
uncertainty robustly. Consequently, we present a thorough overview of so-called
uncertainty models that exhibit uncertainty in a more robust interpretation. We
show several solution techniques for both discrete and continuous models,
ranging from formal verification, over control-based abstractions, to
reinforcement learning. As an integral part of this paper, we list and discuss
several key challenges that arise when dealing with rich types of uncertainty
in a model-based fashion
Efficient Sensitivity Analysis for Parametric Robust Markov Chains
We provide a novel method for sensitivity analysis of parametric robust
Markov chains. These models incorporate parameters and sets of probability
distributions to alleviate the often unrealistic assumption that precise
probabilities are available. We measure sensitivity in terms of partial
derivatives with respect to the uncertain transition probabilities regarding
measures such as the expected reward. As our main contribution, we present an
efficient method to compute these partial derivatives. To scale our approach to
models with thousands of parameters, we present an extension of this method
that selects the subset of parameters with the highest partial derivative.
Our methods are based on linear programming and differentiating these programs
around a given value for the parameters. The experiments show the applicability
of our approach on models with over a million states and thousands of
parameters. Moreover, we embed the results within an iterative learning scheme
that profits from having access to a dedicated sensitivity analysis.Comment: To be presented at CAV 202
CTMCs with Imprecisely Timed Observations
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters but may be uncertain. Thus, we consider a setting in which we are given a sequence of imprecisely timed labels called the evidence. The problem is to compute reachability probabilities, which we condition on this evidence. Our key contribution is a method that solves this problem by unfolding the CTMC states over all possible timings for the evidence. We formalize this unfolding as a Markov decision process (MDP) in which each timing for the evidence is reflected by a scheduler. This MDP has infinitely many states and actions in general, making a direct analysis infeasible. Thus, we abstract the continuous MDP into a finite interval MDP (iMDP) and develop an iterative refinement scheme to upper-bound conditional probabilities in the CTMC. We show the feasibility of our method on several numerical benchmarks and discuss key challenges to further enhance the performance
Efficient Sensitivity Analysis for Parametric Robust Markov Chains
We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are available. We measure sensitivity in terms of partial derivatives with respect to the uncertain transition probabilities regarding measures such as the expected reward. As our main contribution, we present an efficient method to compute these partial derivatives. To scale our approach to models with thousands of parameters, we present an extension of this method that selects the subset of parameters with the highest partial derivative. Our methods are based on linear programming and differentiating these programs around a given value for the parameters. The experiments show the applicability of our approach on models with over a million states and thousands of parameters. Moreover, we embed the results within an iterative learning scheme that profits from having access to a dedicated sensitivity analysis
CTMCs with Imprecisely Timed Observations
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters but may be uncertain. Thus, we consider a setting in which we are given a sequence of imprecisely timed labels called the evidence. The problem is to compute reachability probabilities, which we condition on this evidence. Our key contribution is a method that solves this problem by unfolding the CTMC states over all possible timings for the evidence. We formalize this unfolding as a Markov decision process (MDP) in which each timing for the evidence is reflected by a scheduler. This MDP has infinitely many states and actions in general, making a direct analysis infeasible. Thus, we abstract the continuous MDP into a finite interval MDP (iMDP) and develop an iterative refinement scheme to upper-bound conditional probabilities in the CTMC. We show the feasibility of our method on several numerical benchmarks and discuss key challenges to further enhance the performance