POMDP solving: what rewards do you really expect at execution?

Partially Observable Markov Decision Processes have gained an increasing interest in many research communities, due to sensible improvements of their optimization algorithms and of computers capabilities. Yet, most research focus on optimizing either average accumulated rewards (AI planning) or direct entropy (active perception), whereas none of them matches the rewards actually gathered at execution. Indeed, the first optimization criterion linearly averages over all belief states, so that it does not gain best information from different observations, while the second one totally discards rewards. Thus, motivated by simple demonstrative examples, we study an additive combination of these two criteria to get the best of reward gathering and information acquisition at execution. We then compare our criterion with classical ones, and highlight the need to consider new hybrid non-linear criteria, on a realistic multi-target recognition and tracking mission

Kamerajärjestelmän suunnan optimointi navigointitehtävässä

Navigation in an unknown environment consists of multiple separable subtasks, such as collecting information about the surroundings and navigating to the current goal. In the case of pure visual navigation, all these subtasks need to utilize the same vision system, and therefore a way to optimally control the direction of focus is needed. This thesis presents a case study, where the active sensing problem of directing the gaze of a mobile robot with three machine vision cameras is modeled as a partially observable Markov decision process (POMDP) using a mutual information (MI) based reward function. The key aspect of the solution is that the cameras are dynamically used either in monocular or stereo configuration. The algorithms are implemented on Robot Operating System (ROS) and the benefits of using the proposed active sensing implementation over fixed stereo cameras are demonstrated with simulations experiments. The proposed active sensing outperforms the fixed camera solution when prior information about the environment is highly uncertain, and performs just as good in other tested scenarios. --- Navigaatio ennalta tuntemattomassa ympäristössä koostuu useista erillisistä alitehtävistä kuten informaation keräämisestä ja tämänhetkiseen kohteeseen navigoinnista. Kun kyse on puhtaasti visuaalisesta navigoinnista, tarvitsee kaikkien alitehtävien hyödyntää samaa kamerajärjestelmää, joten kamerajärjestelmän suunnan optimointi on tarpeen. Tässä diplomityössä esitellään esimerkkitapaus, jossa kolmen mobiiliin robottiin kiinnitetyn kameran suunnan aktiivinen operointiongelma mallinnetaan osittain havaittavana Markov-päätösprosessina (POMDP), jossa käytetään keskinäisinformaatioon (MI) perustuvaa palkkiota. Olennainen osa ratkaisua on, että kameroita voidaan käyttää dynaamisesti sekä monokulaarisessa- että stereokamera-konfiguraatiossa. Kehitetyt algoritmit implementoidaan Robot Operating System (ROS) -järjestelmälle ja kameroiden aktiivisen operoinnin hyödyt verrattuna kiinteästi asennettuihin stereokameroihin osoitetaan simulaatioilla. Kehitetty aktiivinen operointi suoriutuu kiinteitä kameroita paremmin kun ennakkotieto ympäristöstä on hyvin epävarmaa, ja muissa kokeilluissa tapauksissa vähintään yhtä hyvin

Active Sensing for Partially Observable Markov Decision Processes

Context information on a smart phone can be used to tailor applications for specific situations (e.g. provide tailored routing advice based on location, gas prices and traffic). However, typical context-aware smart phone applications use very limited context information such as user identity, location and time. In the future, smart phones will need to decide from a wide range of sensors to gather information from in order to best accommodate user needs and preferences in a given context. In this thesis, we present a model for active sensor selection within decision-making processes, in which observational features are selected based on longer-term impact on the decisions made by the smart phone. This thesis formulates the problem as a partially observable Markov decision process (POMDP), and proposes a non-myopic solution to the problem using a state of the art approximate planning algorithm Symbolic Perseus. We have tested our method on a 3 small example domains, comparing different policy types, discount factors and cost settings. The experimental results proved that the proposed approach delivers a better policy in the situation of costly sensors, while at the same time provides the advantage of faster policy computation with less memory usage

A Decision-Theoretic Approach to Dynamic Sensor Selection in Camera Networks

Nowadays many urban areas have been equipped with networks of surveillance cameras, which can be used for automatic localization and tracking of people. However, given the large resource demands of imaging sensors in terms of bandwidth and computing power, processing the image streams of all cameras simultaneously might not be feasible. In this paper, we consider the problem of dynamical sensor selection based on user-defined objectives, such as maximizing coverage or improved localization uncertainty.  We propose a decision-theoretic approach modeled as a POMDP, which selects k sensors to consider in the next time frame, incorporating all observations made in the past. We show how, by changing the POMDP's reward function, we can change the system's behavior in a straightforward manner, fulfilling the user's chosen objective. We successfully apply our techniques to a network of 10 cameras