3 research outputs found
Distinguishing Hidden Markov Chains
Hidden Markov Chains (HMCs) are commonly used mathematical models of
probabilistic systems. They are employed in various fields such as speech
recognition, signal processing, and biological sequence analysis. We consider
the problem of distinguishing two given HMCs based on an observation sequence
that one of the HMCs generates. More precisely, given two HMCs and an
observation sequence, a distinguishing algorithm is expected to identify the
HMC that generates the observation sequence. Two HMCs are called
distinguishable if for every there is a distinguishing
algorithm whose error probability is less than . We show that one
can decide in polynomial time whether two HMCs are distinguishable. Further, we
present and analyze two distinguishing algorithms for distinguishable HMCs. The
first algorithm makes a decision after processing a fixed number of
observations, and it exhibits two-sided error. The second algorithm processes
an unbounded number of observations, but the algorithm has only one-sided
error. The error probability, for both algorithms, decays exponentially with
the number of processed observations. We also provide an algorithm for
distinguishing multiple HMCs. Finally, we discuss an application in stochastic
runtime verification.Comment: This is the full version of a LICS'16 pape
Formal methods paradigms for estimation and machine learning in dynamical systems
Formal methods are widely used in engineering to determine whether a system exhibits a certain property (verification) or to design controllers that are guaranteed to drive the system to achieve a certain property (synthesis). Most existing techniques require a large amount of accurate information about the system in order to be successful. The methods presented in this work can operate with significantly less prior information. In the domain of formal synthesis for robotics, the assumptions of perfect sensing and perfect knowledge of system dynamics are unrealistic. To address this issue, we present control algorithms that use active estimation and reinforcement learning to mitigate the effects of uncertainty. In the domain of cyber-physical system analysis, we relax the assumption that the system model is known and identify system properties automatically from execution data.
First, we address the problem of planning the path of a robot under temporal logic constraints (e.g. "avoid obstacles and periodically visit a recharging station") while simultaneously minimizing the uncertainty about the state of an unknown feature of the environment (e.g. locations of fires after a natural disaster). We present synthesis algorithms and evaluate them via simulation and experiments with aerial robots. Second, we develop a new specification language for tasks that require gathering information about and interacting with a partially observable environment, e.g. "Maintain localization error below a certain level while also avoiding obstacles.'' Third, we consider learning temporal logic properties of a dynamical system from a finite set of system outputs. For example, given maritime surveillance data we wish to find the specification that corresponds only to those vessels that are deemed law-abiding. Algorithms for performing off-line supervised and unsupervised learning and on-line supervised learning are presented. Finally, we consider the case in which we want to steer a system with unknown dynamics to satisfy a given temporal logic specification. We present a novel reinforcement learning paradigm to solve this problem. Our procedure gives "partial credit'' for executions that almost satisfy the specification, which can
lead to faster convergence rates and produce better solutions when the specification is not satisfiable
Predictive Runtime Verification of Stochastic Systems
Runtime Verification (RV) is the formal analysis of the execution of a system against some
properties at runtime. RV is particularly useful for stochastic systems that have a non-zero
probability of failure at runtime. The standard RV assumes constructing a monitor that
checks only the currently observed execution of the system against the given properties.
This dissertation proposes a framework for predictive RV, where the monitor instead
checks the current execution with its finite extensions against some property. The extensions are generated using a prediction model, that is built based on execution samples
randomly generated from the system. The thesis statement is that predictive RV for
stochastic systems is feasible, effective, and useful.
The feasibility is demonstrated by providing a framework, called Prevent, that builds a
predictive monitor by using trained prediction models to finitely extend an execution path,
and computing the probabilities of the extensions that satisfy or violate the given property.
The prediction model is trained using statistical learning techniques from independent and
identically distributed samples of system executions. The prediction is the result of a
quantitative bounded reachability analysis on the product of the prediction model and the
automaton specifying the property. The analysis results are computed offline and stored in
a lookup table. At runtime the monitor obtains the state of the system on the prediction
model based on the observed execution, directly or by approximation, and uses the lookup
table to retrieve the computed probability that the system at the current state will satisfy
or violate the given property within some finite number of steps.
The effectiveness of Prevent is shown by applying abstraction when constructing the
prediction model. The abstraction is on the observation space based on extracting the
symmetry relation between symbols that have similar probabilities to satisfy a property.
The abstraction may introduce nondeterminism in the final model, which is handled by
using a hidden state variable when building the prediction model. We also demonstrate
that, under the convergence conditions of the learning algorithms, the prediction results
from the abstract models are the same as the concrete models.
Finally, the usefulness of Prevent is indicated in real-world applications by showing
how it can be applied for predicting rare properties, properties with very low but non-zero
probability of satisfaction. More specifically, we adjust the training algorithm that uses
the samples generated by importance sampling to generate the prediction models for rare
properties without increasing the number of samples and without having a negative impact
on the prediction accuracy