5 research outputs found
Revisiting Weak Simulation for Substochastic Markov Chains
Contains fulltext :
117207.pdf (preprint version ) (Closed access
Technical Report: Distribution Temporal Logic: Combining Correctness with Quality of Estimation
We present a new temporal logic called Distribution Temporal Logic (DTL)
defined over predicates of belief states and hidden states of partially
observable systems. DTL can express properties involving uncertainty and
likelihood that cannot be described by existing logics. A co-safe formulation
of DTL is defined and algorithmic procedures are given for monitoring
executions of a partially observable Markov decision process with respect to
such formulae. A simulation case study of a rescue robotics application
outlines our approach.Comment: More expanded version of "Distribution Temporal Logic: Combining
Correctness with Quality of Estimation" to appear in IEEE CDC 201
Probabilistic Bisimulation: Naturally on Distributions
In contrast to the usual understanding of probabilistic systems as stochastic
processes, recently these systems have also been regarded as transformers of
probabilities. In this paper, we give a natural definition of strong
bisimulation for probabilistic systems corresponding to this view that treats
probability distributions as first-class citizens. Our definition applies in
the same way to discrete systems as well as to systems with uncountable state
and action spaces. Several examples demonstrate that our definition refines the
understanding of behavioural equivalences of probabilistic systems. In
particular, it solves a long-standing open problem concerning the
representation of memoryless continuous time by memory-full continuous time.
Finally, we give algorithms for computing this bisimulation not only for finite
but also for classes of uncountably infinite systems
Belief Bisimulation for Hidden Markov Models Logical Characterisation and Decision Algorithm
Contains fulltext :
93765.pdf (publisher's version ) (Closed access
Formal methods paradigms for estimation and machine learning in dynamical systems
Formal methods are widely used in engineering to determine whether a system exhibits a certain property (verification) or to design controllers that are guaranteed to drive the system to achieve a certain property (synthesis). Most existing techniques require a large amount of accurate information about the system in order to be successful. The methods presented in this work can operate with significantly less prior information. In the domain of formal synthesis for robotics, the assumptions of perfect sensing and perfect knowledge of system dynamics are unrealistic. To address this issue, we present control algorithms that use active estimation and reinforcement learning to mitigate the effects of uncertainty. In the domain of cyber-physical system analysis, we relax the assumption that the system model is known and identify system properties automatically from execution data.
First, we address the problem of planning the path of a robot under temporal logic constraints (e.g. "avoid obstacles and periodically visit a recharging station") while simultaneously minimizing the uncertainty about the state of an unknown feature of the environment (e.g. locations of fires after a natural disaster). We present synthesis algorithms and evaluate them via simulation and experiments with aerial robots. Second, we develop a new specification language for tasks that require gathering information about and interacting with a partially observable environment, e.g. "Maintain localization error below a certain level while also avoiding obstacles.'' Third, we consider learning temporal logic properties of a dynamical system from a finite set of system outputs. For example, given maritime surveillance data we wish to find the specification that corresponds only to those vessels that are deemed law-abiding. Algorithms for performing off-line supervised and unsupervised learning and on-line supervised learning are presented. Finally, we consider the case in which we want to steer a system with unknown dynamics to satisfy a given temporal logic specification. We present a novel reinforcement learning paradigm to solve this problem. Our procedure gives "partial credit'' for executions that almost satisfy the specification, which can
lead to faster convergence rates and produce better solutions when the specification is not satisfiable