31,826 research outputs found
Model checking Quantitative Linear Time Logic
This paper considers QLtl, a quantitative analagon of Ltl and presents algorithms for model checking QLtl over quantitative versions of Kripke structures and Markov chains
The Complexity of POMDPs with Long-run Average Objectives
We study the problem of approximation of optimal values in
partially-observable Markov decision processes (POMDPs) with long-run average
objectives. POMDPs are a standard model for dynamic systems with probabilistic
and nondeterministic behavior in uncertain environments. In long-run average
objectives rewards are associated with every transition of the POMDP and the
payoff is the long-run average of the rewards along the executions of the
POMDP. We establish strategy complexity and computational complexity results.
Our main result shows that finite-memory strategies suffice for approximation
of optimal values, and the related decision problem is recursively enumerable
complete
Qualitative Analysis of POMDPs with Temporal Logic Specifications for Robotics Applications
We consider partially observable Markov decision processes (POMDPs), that are
a standard framework for robotics applications to model uncertainties present
in the real world, with temporal logic specifications. All temporal logic
specifications in linear-time temporal logic (LTL) can be expressed as parity
objectives. We study the qualitative analysis problem for POMDPs with parity
objectives that asks whether there is a controller (policy) to ensure that the
objective holds with probability 1 (almost-surely). While the qualitative
analysis of POMDPs with parity objectives is undecidable, recent results show
that when restricted to finite-memory policies the problem is EXPTIME-complete.
While the problem is intractable in theory, we present a practical approach to
solve the qualitative analysis problem. We designed several heuristics to deal
with the exponential complexity, and have used our implementation on a number
of well-known POMDP examples for robotics applications. Our results provide the
first practical approach to solve the qualitative analysis of robot motion
planning with LTL properties in the presence of uncertainty
Probabilistic Bisimulations for PCTL Model Checking of Interval MDPs
Verification of PCTL properties of MDPs with convex uncertainties has been
investigated recently by Puggelli et al. However, model checking algorithms
typically suffer from state space explosion. In this paper, we address
probabilistic bisimulation to reduce the size of such an MDPs while preserving
PCTL properties it satisfies. We discuss different interpretations of
uncertainty in the models which are studied in the literature and that result
in two different definitions of bisimulations. We give algorithms to compute
the quotients of these bisimulations in time polynomial in the size of the
model and exponential in the uncertain branching. Finally, we show by a case
study that large models in practice can have small branching and that a
substantial state space reduction can be achieved by our approach.Comment: In Proceedings SynCoP 2014, arXiv:1403.784
Bellman Error Based Feature Generation using Random Projections on Sparse Spaces
We address the problem of automatic generation of features for value function
approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve
the error of policy evaluation with function approximation, with a convergence
rate similar to that of value iteration. We propose a simple, fast and robust
algorithm based on random projections to generate BEBFs for sparse feature
spaces. We provide a finite sample analysis of the proposed method, and prove
that projections logarithmic in the dimension of the original space are enough
to guarantee contraction in the error. Empirical results demonstrate the
strength of this method
- …