1 research outputs found
Active Trajectory Estimation for Partially Observed Markov Decision Processes via Conditional Entropy
In this paper, we consider the problem of controlling a partially observed
Markov decision process (POMDP) in order to actively estimate its state
trajectory over a fixed horizon with minimal uncertainty. We pose a novel
active smoothing problem in which the objective is to directly minimise the
smoother entropy, that is, the conditional entropy of the (joint) state
trajectory distribution of concern in fixed-interval Bayesian smoothing. Our
formulation contrasts with prior active approaches that minimise the sum of
conditional entropies of the (marginal) state estimates provided by Bayesian
filters. By establishing a novel form of the smoother entropy in terms of the
POMDP belief (or information) state, we show that our active smoothing problem
can be reformulated as a (fully observed) Markov decision process with a value
function that is concave in the belief state. The concavity of the value
function is of particular importance since it enables the approximate solution
of our active smoothing problem using piecewise-linear function approximations
in conjunction with standard POMDP solvers. We illustrate the approximate
solution of our active smoothing problem in simulation and compare its
performance to alternative approaches based on minimising marginal state
estimate uncertainties.Comment: 7 pages, 3 figures, accepted for presentation at 2021 European
Control Conferenc