Target Search Planning in Uncertain Environments

The autonomous robots are useful for lot of things, such as rescue in dangerous environments. In this thesis, we consider how autonomous robots, the Unmanned Aerial Vehicles (UAVs), make a plan to travel in an indoor uncertain environment. At the same time, the robots will observe and update the environment representations with their on-board sensors and plan the path for each robot in the robot group. They will avoid collisions and cooperate with others in the Complete Mission Process (CMP), which includes all operations of robots before the mission is completed (all targets are visited). The environment cannot be represented exactly because of the inaccurate representation model and the sensor noises. In order to complete the mission efficiently, single robot requires a method to plan a path for efficient travelling from a start point to a target point, plan an assignment for visiting all its targets one by one. For multiple robots in a robot group, we need to plan an allocation for allocating multiple targets to multiple robots in order to make sure that all robots can cooperate together. All these planning operations have to be done based on an inexact representation of the environment. This thesis focuses on the path/assignment/allocation planning problem in environments which are not completely known, based on a reduced/simplified —Partially Observable Markov Decision Process (POMDP) — framework. The former researches only consider the initial plan but neglect the later replans. Our approach considers the plan and the re-plans from the start to the completion of the mission. Our novel Monte Carlo based planning approaches will plan a path for one robot to move efficiently from one point to one target, plan an assignment for one robot to visit multiple targets by travelling the shortest route and plan an allocation for multiple robots to cooperate and visit multiple targets as soon as possible (the planning time plus the travelling time is minimized). Our approach is based on a Monte Carlo sampling strategy. In order to decrease its computational cost, two strategies are proposed. We then extend our approach to multiple robots and multiple targets scenario. Finally, the approaches are extended to multiple robots and multiple targets scenario. They are characterised and evaluated experimentally through simulation. When we compare it with similar methods from the literatures, our approach can provide the better solution

Lossless clustering of histories in decentralized POMDPs

Decentralized partially observable Markov decision processes (Dec-POMDPs) constitute a generic and expressive framework for multiagent planning under uncertainty. However, planning optimally is difficult because solutions map local observation histories to actions, and the number of such histories grows exponentially in the planning horizon. In this work, we identify a criterion that allows for lossless clustering of observation histories: i.e., we prove that when two histories satisfy the criterion, they have the same optimal value and thus can be treated as one. We show how this result can be exploited in optimal policy search and demonstrate empirically that it can provide a speed-up of multiple orders of magnitude, allowing the optimal solution of significantly larger problems. We also perform an empirical analysis of the generality of our clustering method, which suggests that it may also be useful in other (approximate) Dec-POMDP solution methods