Analysis of methods for playing human robot hide-and-seek in a simple real world urban environment

The hide-and-seek game has many interesting aspects for studying cognitive functions in robots and the interactions between mobile robots and humans. Some MOMDP (Mixed Observable Markovian Decision Processes) models and a heuristic-based method are proposed and evaluated as an automated seeker. MOMDPs are used because the hider's position is not always known (partially observable), and the seeker's position is fully observable. The MOMDP model is used in an o-line method for which two reward functions are tried. Because the time complexity of this model grows exponentially with the number of (partially observable) states, an on-line hierarchical MOMDP model was proposed to handle bigger maps. To reduce the states in the on-line method a robot centered segmentation is used. In addition to extensive simulations, games with a human hider and a real mobile robot as a seeker have been done in a simple urban environment.Peer ReviewedPostprint (author’s final draft

A POMDP approach to the hide and seek game

Projecte final de Màster Oficial fet en col.laboració amb Institut de Robàtica i Informàtica IndustrialPartially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in uncertain and dynamic environments. They have been successfully applied to various robotic tasks. The modeling advantage of POMDPs, however, comes at a price exact methods for solving them are computationally very expensive and thus applicable in practice only to simple problems. A major challenge is to scale up POMDP algorithms for more complex robotic systems. Our goal is to make an autonomous mobile robot to learn and play the children's game hide and seek with opponent a human agent. Motion planning in uncertain and dynamic envi- ronments is an essential capability for autonomous robots. We focus on an e cient point-based POMDP algorithm, SARSOP, that exploits the notion of optimally reachable belief spaces to improve computational efficiency. Moreover we explore the mixed observability MDPs (MOMDPs) model, a special class of POMDPs. Robotic systems often have mixed observability: even when a robots state is not fully observable, some components of the state may still be fully observable. Ex- ploiting this, we use the factored model, proposed in the literature, to represent separately the fully and partially observable components of a robots state and derive a compact lower dimensional representation of its belief space. We then use this factored representation in conjunction with the point-based algorithm to com- pute approximate POMDP solutions. Experiments show that on our problem, the new algorithm is many times faster than a leading point-based POMDP algorithm without important losses in the quality of the solutio

Continuous real time POMCP to find-and-follow people by a humanoid service robot

Trabajo presentado al 14th IEEE-RAS International Conference on Humanoid Robots: Humanoids 2014 "Humans and Robots Face-to-Face", celebrado en Madrid (España) del 18 al 20 de noviembre de 2014.This study describes and evaluates two new methods for finding and following people in urban settings using a humanoid service robot: the Continuous Real-time POMCP method, and its improved extension called Adaptive Highest Belief Continuous Real-time POMCP follower. They are able to run in real-time, in large continuous environments. These methods make use of the online search algorithm Partially Observable Monte-Carlo Planning (POMCP), which in contrast to other previous approaches, can plan under uncertainty on large state spaces. We compare our new methods with a heuristic person follower and demonstrate that they obtain better results by testing them extensively in both simulated and real-life experiments. More than two hours, over 3 km, of autonomous navigation during real-life experiments have been done with a mobile humanoid robot in urban environments.This work has been partially funded by the DPI2013-42458-P.Peer Reviewe

Searching and tracking people with cooperative mobile robots

The final publication is available at link.springer.comSocial robots should be able to search and track people in order to help them. In this paper we present two different techniques for coordinated multi-robot teams for searching and tracking people. A probability map (belief) of a target person location is maintained, and to initialize and update it, two methods were implemented and tested: one based on a reinforcement learning algorithm and the other based on a particle filter. The person is tracked if visible, otherwise an exploration is done by making a balance, for each candidate location, between the belief, the distance, and whether close locations are explored by other robots of the team. The validation of the approach was accomplished throughout an extensive set of simulations using up to five agents and a large amount of dynamic obstacles; furthermore, over three hours of real-life experiments with two robots searching and tracking were recorded and analysed.Peer ReviewedPostprint (author's final draft

Comparison of MOMDP and heuristic methods to play hide-and-seek

Trabajo presentado al 16th International Conference of the Catalan Association for Artificial Intelligence en Vic del 23 al 25 de octubre de 2013.The hide-and-seek game is considered an excellent domain for studying the interactions between mobile robots and humans. Prior to the implementation and test in our mobile robots TIBI and DABO, we have been devising different models and strategies to play this game and comparing them extensively in simulations. Recently, we have proposed the use of MOMDP (Mixed Observability Markov Decision Processes) models to learn a good policy to be applied by the seeker. Even though MOMDPs reduce the computational cost of POMDPs (Partially Observable MDPs), they still have a high computational complexity which is exponential with the number of states. For the hide-and-seek game, the number of states is directly related to the number of grid cells, and for two players (the hider and the seeker), it is the square of the number of cells. As an alternative to off-line MOMDP policy computation with the complete grid fine resolution, we have devised a two-level MOMDP, where the policy is computed on-line at the top level with a reduced number of states independent of the grid size. In this paper, we introduce a new fast heuristic method for the seeker and compare its performance to both off-line and on-line MOMDP approaches. We show simulation results in maps of different sizes against two types of automated hiders.Work supported by the Spanish Ministry of Science and Innovation under project Rob- TaskCoop (DPI2010-17112).Peer Reviewe