20,338 research outputs found
Stochastic Shortest Path with Energy Constraints in POMDPs
We consider partially observable Markov decision processes (POMDPs) with a
set of target states and positive integer costs associated with every
transition. The traditional optimization objective (stochastic shortest path)
asks to minimize the expected total cost until the target set is reached. We
extend the traditional framework of POMDPs to model energy consumption, which
represents a hard constraint. The energy levels may increase and decrease with
transitions, and the hard constraint requires that the energy level must remain
positive in all steps till the target is reached. First, we present a novel
algorithm for solving POMDPs with energy levels, developing on existing POMDP
solvers and using RTDP as its main method. Our second contribution is related
to policy representation. For larger POMDP instances the policies computed by
existing solvers are too large to be understandable. We present an automated
procedure based on machine learning techniques that automatically extracts
important decisions of the policy allowing us to compute succinct human
readable policies. Finally, we show experimentally that our algorithm performs
well and computes succinct policies on a number of POMDP instances from the
literature that were naturally enhanced with energy levels.Comment: Technical report accompanying a paper published in proceedings of
AAMAS 201
Parameter Identification in a Probabilistic Setting
Parameter identification problems are formulated in a probabilistic language,
where the randomness reflects the uncertainty about the knowledge of the true
values. This setting allows conceptually easily to incorporate new information,
e.g. through a measurement, by connecting it to Bayes's theorem. The unknown
quantity is modelled as a (may be high-dimensional) random variable. Such a
description has two constituents, the measurable function and the measure. One
group of methods is identified as updating the measure, the other group changes
the measurable function. We connect both groups with the relatively recent
methods of functional approximation of stochastic problems, and introduce
especially in combination with the second group of methods a new procedure
which does not need any sampling, hence works completely deterministically. It
also seems to be the fastest and more reliable when compared with other
methods. We show by example that it also works for highly nonlinear non-smooth
problems with non-Gaussian measures.Comment: 29 pages, 16 figure
Quantitative Regular Expressions for Arrhythmia Detection Algorithms
Motivated by the problem of verifying the correctness of arrhythmia-detection
algorithms, we present a formalization of these algorithms in the language of
Quantitative Regular Expressions. QREs are a flexible formal language for
specifying complex numerical queries over data streams, with provable runtime
and memory consumption guarantees. The medical-device algorithms of interest
include peak detection (where a peak in a cardiac signal indicates a heartbeat)
and various discriminators, each of which uses a feature of the cardiac signal
to distinguish fatal from non-fatal arrhythmias. Expressing these algorithms'
desired output in current temporal logics, and implementing them via monitor
synthesis, is cumbersome, error-prone, computationally expensive, and sometimes
infeasible.
In contrast, we show that a range of peak detectors (in both the time and
wavelet domains) and various discriminators at the heart of today's
arrhythmia-detection devices are easily expressible in QREs. The fact that one
formalism (QREs) is used to describe the desired end-to-end operation of an
arrhythmia detector opens the way to formal analysis and rigorous testing of
these detectors' correctness and performance. Such analysis could alleviate the
regulatory burden on device developers when modifying their algorithms. The
performance of the peak-detection QREs is demonstrated by running them on real
patient data, on which they yield results on par with those provided by a
cardiologist.Comment: CMSB 2017: 15th Conference on Computational Methods for Systems
Biolog
Finite-state Strategies in Delay Games (full version)
What is a finite-state strategy in a delay game? We answer this surprisingly
non-trivial question by presenting a very general framework that allows to
remove delay: finite-state strategies exist for all winning conditions where
the resulting delay-free game admits a finite-state strategy. The framework is
applicable to games whose winning condition is recognized by an automaton with
an acceptance condition that satisfies a certain aggregation property. Our
framework also yields upper bounds on the complexity of determining the winner
of such delay games and upper bounds on the necessary lookahead to win the
game. In particular, we cover all previous results of that kind as special
cases of our uniform approach
- …