Particle filter approximations for general open loop and open loop feedback sensor management

Sensor management is a stochastic control problem where the control mechanism is directed at the generation of observations. Typically, sensor management attempts to optimize a certain statistic derived from the posterior distribution of the state, such as covariance or entropy. However, these statistics often depend on future measurements which are not available at the moment the control decision is taken, making it necessary to consider their expectation over the entire measurement space. Though the idea of computing such expectations using a particle filter is not new, so far it has been applied only to specific sensor management problems and criterions. In this memorandum, for a considerably broad class of problems, we explicitly show how particle filters can be used to approximate general sensor management criterions in the open loop and open loop feedback cases. As examples, we apply these approximations to selected sensor management criterions. As an additional contribution of this memorandum, we show that every performance metric can be used to define a corresponding estimate and a corresponding task-driven sensor management criterion, and both of them can be approximated using particle filters. This is used to propose an approximate sensor management scheme based on the OSPA metric for multi-target tracking, which is included among our examples

Active Sensing for Partially Observable Markov Decision Processes

Context information on a smart phone can be used to tailor applications for specific situations (e.g. provide tailored routing advice based on location, gas prices and traffic). However, typical context-aware smart phone applications use very limited context information such as user identity, location and time. In the future, smart phones will need to decide from a wide range of sensors to gather information from in order to best accommodate user needs and preferences in a given context. In this thesis, we present a model for active sensor selection within decision-making processes, in which observational features are selected based on longer-term impact on the decisions made by the smart phone. This thesis formulates the problem as a partially observable Markov decision process (POMDP), and proposes a non-myopic solution to the problem using a state of the art approximate planning algorithm Symbolic Perseus. We have tested our method on a 3 small example domains, comparing different policy types, discount factors and cost settings. The experimental results proved that the proposed approach delivers a better policy in the situation of costly sensors, while at the same time provides the advantage of faster policy computation with less memory usage

Probabilistic Framework for Sensor Management

A probabilistic sensor management framework is introduced, which maximizes the utility of sensor systems with many different sensing modalities by dynamically configuring the sensor system in the most beneficial way. For this purpose, techniques from stochastic control and Bayesian estimation are combined such that long-term effects of possible sensor configurations and stochastic uncertainties resulting from noisy measurements can be incorporated into the sensor management decisions

Characterization of uncertainty in Bayesian estimation using sequential Monte Carlo methods

In estimation problems, accuracy of the estimates of the quantities of interest cannot be taken for granted. This means that estimation errors are expected, and a good estimation algorithm should be able not only to compute estimates that are optimal in some sense, but also provide meaningful measures of uncertainty associated with those estimates. In some situations, we might also be able to reduce estimation uncertainty through the use of feedback on observations, an approach referred to as sensor management. Characterization of estimation uncertainty, as well as sensor management, are certainly difficult tasks for general partially observed processes, which might be non-linear, non-Gaussian, and/or have dependent process and observation noises. Sequential Monte Carlo (SMC) methods, also known as particle filters, are numerical Bayesian estimators which are, in principle, able to handle highly general estimation problems. However, SMC methods are known to suffer from a phenomenon called degeneracy, or self-resolving, which greatly impairs their usefulness against certain classes of problems. One of such classes, that we address in the first part of this thesis, is the joint state and parameter estimation problem, where there are unknown parameters to be estimated together with the timevarying state. Some SMC variants have been proposed to counter the degeneracy phenomenon for this problem, but these state-of-the-art techniques are either non-Bayesian or introduce biases on the system model, which might not be appropriate if proper characterization of estimation uncertainty is required. For this type of scenario, we propose using the Rao-Blackwellized Marginal Particle Filter (RBMPF), a combination of two SMC algorithm variants: the Rao-Blackwellized Particle Filter (RBPF) and the Marginal Particle Filter (MPF). We derive two new versions of the RBMPF: one for models with low dimensional parameter vectors, and another for more general models. We apply the proposed methods to two practical problems: the target tracking problem of turn rate estimation for a constant turn maneuver, and the econometrics problem of stochastic volatility estimation. Our proposed methods are shown to be effective solutions, both in terms of estimation accuracy and statistical consistency, i.e. characterization of estimation uncertainty. Another problem where standard particle filters suffer from degeneracy, addressed in the second part of this thesis, is the joint multi-target tracking and labelling problem. In comparison with the joint state and parameter estimation problem, this problem poses an additional challenge, namely, the fact that it has not been properly mathematically formulated in previous literature. Using Finite Set Statistics (FISST), we provide a sound theoretical formulation for the problem, and in order to actually solve the problem, we propose a novel Bayesian algorithm, the Labelling Uncertainty-Aware Particle Filter (LUA-PF) filter, essentially a combination of the RBMPF and the Multi-target Sequential Monte Carlo (M-SMC) filter techniques. We show that the new algorithm achieves significant improvements on both finding the correct track labelling and providing a meaningful measure of labelling uncertainty. In the last part of this thesis, we address the sensor management problem. Although we apply particle filters to the problem, they are not the main focus of this part of the work. Instead, we concentrate on a more fundamental question, namely, which sensor management criterion should be used in order to obtain the best results in terms of information gain and/or reduction of uncertainty. In order to answer this question, we perform an in-depth theoretical and empirical analysis on two popular sensor management criteria based on information theory – the Kullback-Leibler and R´enyi divergences. On the basis of this analysis, we are able to either confirm or reject some previous arguments used as theoretical justification for these two criteria