An extension of GHMMs for environments with occlusions and automatic goal discovery for person trajectory prediction

This work is partially funded by the EC-FP7 under grant agreement no. 611153 (TERESA) and the project PAIS-MultiRobot, funded by the Junta de Andalucía (TIC-7390). I. Perez-Hurtado is also supported by the Postdoctoral Junior Grant 2013 co-funded by the Spanish Ministry of Economy and Competitiveness and the Pablo de Olavide University.Robots navigating in a social way should use some knowledge about common motion patterns of people in the environment. Moreover, it is known that people move intending to reach certain points of interest, and machine learning techniques have been widely used for acquiring this knowledge by observation. Learning algorithms such as Growing Hidden Markov Models (GHMMs) usually assume that points of interest are located at the end of human trajectories, but complete trajectories cannot always be observed by a mobile robot due to occlusions and people going out of sensor range. This paper extends GHMMs to deal with partial observed trajectories where people's goals are not known a priori. A novel technique based on hypothesis testing is also used to discover the points of interest (goals) in the environment. The approach is validated by predicting people's motion in three different datasets.Universidad Pablo de Olavide. Departamento de Deporte e InformáticaPostprin

Building Prior Knowledge: A Markov Based Pedestrian Prediction Model Using Urban Environmental Data

International audienceAutonomous Vehicles navigating in urban areas have a need to understand and predict future pedestrian behavior for safer navigation. This high level of situational awareness requires observing pedestrian behavior and extrapolating their positions to know future positions. While some work has been done in this field using Hidden Markov Models (HMMs), one of the few observed drawbacks of the method is the need for informed priors for learning behavior. In this work, an extension to the Growing Hidden Markov Model (GHMM) method is proposed to solve some of these drawbacks. This is achieved by building on existing work using potential cost maps and the principle of Natural Vision. As a consequence, the proposed model is able to predict pedestrian positions more precisely over a longer horizon compared to the state of the art. The method is tested over "legal" and "illegal" behavior of pedestrians, having trained the model with sparse observations and partial trajectories. The method, with no training data, is compared against a trained state of the art model. It is observed that the proposed method is robust even in new, previously unseen areas

Acquisition et exploitation des connaissances antérieures pour prédire le comportement des piétons autour des véhicules autonomes en environnement urbain

Autonomous Vehicles navigating in urban areas interact with pedestrians and other shared space users like bicycles throughout their journey either in open areas, like urban city centers, or closed areas, like parking lots. As more and more autonomous vehicles take to the city streets, their ability to understand and predict pedestrian behaviour becomes paramount. This is achieved by learning through continuous observation of the area to drive in. On the other hand, human drivers can instinctively infer pedestrian motion on an urban street even in previously unseen areas. This need for increasing a vehicle's situational awareness to reach parity with human drivers fuels the need for larger and deeper data on pedestrian motion in myriad situations and varying environments.This thesis focuses on the problem of reducing this dependency on large amounts of data to predict pedestrian motion accurately over an extended horizon. Instead, this work relies on Prior Knowledge, itself derived from the JJ Gibson's sociological principles of ``Natural Vision'' and ``Natural Movement''. It assumes that pedestrian behaviour is a function of the built environment and that all motion is directed towards reaching a goal. Knowing this underlying principle, the cost for traversing a scene from a pedestrian's perspective can be divined. Knowing this, inference on their behaviour can be performed. This work presents a contribution to the framework of understanding pedestrian behaviour as a confluence of probabilistic graphical models and sociological principles in three ways: modelling the environment, learning and predicting.Concerning modelling, the work assumes that there are some parts of the observed scene which are more attractive to pedestrians and some areas, repulsive. By quantifying these ``affordances'' as a consequence of certain Points of Interest (POIs) and the different elements in the scene, it is possible to model this scene under observation with different costs as a basis of the features contained within.Concerning learning, this work primarily extends the Growing Hidden Markov Model (GHMM) method - a variant of the Hidden Markov Model (HMM) probabilistic model- with the application of Prior Knowledge to initialise a topology able to infer accurately on ``typical motions'' in the scene. Secondly, the model that is generated behaves as a Self-Organising map, incrementally learning non-typical pedestrian behaviour and encoding this within the topology while updating the parameters of the underlying HMM.On prediction, this work carries out Bayesian inference on the generated model and can, as a result of Prior Knowledge, manage to perform better than the existing implementation of the GHMM method in predicting future pedestrian positions without the availability of training trajectories, thereby allowing for its utilisation in an urban scene with only environmental data.The contributions of this thesis are validated through experimental results on real data captured from an overhead camera overlooking a busy urban street, depicting a structured built environment and from the car's perspective in a parking lot, depicting a semi-structured environment and tested on typical and non-typical trajectories in each case.Les véhicules autonomes qui naviguent dans les zones urbaines interagissent avec les piétons et les autres utilisateurs de l'espace partagé, comme les bicyclettes, tout au long de leur trajet, soit dans des zones ouvertes, comme les centres urbains, soit dans des zones fermées, comme les parcs de stationnement. Alors que de plus en plus de véhicules autonomes sillonnent les rues de la ville, leur capacité à comprendre et à prévoir le comportement des piétons devient primordiale. Ceci est possible grâce à l'apprentissage par l'observation continue de la zone à conduire. D'autre part, les conducteurs humains peuvent instinctivement déduire le mouvement des piétons sur une rue urbaine, même dans des zones auparavant invisibles. Ce besoin d'accroître la conscience de la situation d'un véhicule pour atteindre la parité avec les conducteurs humains alimente le besoin de données plus vastes et plus approfondies sur le mouvement des piétons dans une myriade de situations et d'environnements variés.Cette thèse porte sur le problème de la réduction de cette dépendance à l'égard de grandes quantités de données pour prédire avec précision les mouvements des piétons sur un horizon prolongé. Ce travail s'appuie plutôt sur la connaissance préalable, elle-même dérivée des principes sociologiques de "Vision naturelle" et de "Mouvement naturel" du JJ Gibson. Il suppose que le comportement des piétons est fonction de l'environnement bâti et que tous les mouvements sont orientés vers l'atteinte d'un but. Connaissant ce principe sous-jacent, le coût de la traversée d'une scène du point de vue d'un piéton peut être deviné. Sachant cela, on peut en déduire leur comportement. Cet ouvrage apporte une contribution au cadre de compréhension du comportement piétonnier en tant que confluent de modèles graphiques probabilistes et de principes sociologiques de trois façons : modélisation de l'environnement, apprentissage et prévision.En ce qui concerne la modélisation, le travail suppose que certaines parties de la scène observée sont plus attrayantes pour les piétons et que d'autres sont répugnantes. En quantifiant ces " affordances " en fonction de certains Points d'Intérêt (POI) et des différents éléments de la scène, il est possible de modéliser cette scène sous observation avec différents coûts comme base des caractéristiques qu'elle contient.En ce qui concerne l'apprentissage, ce travail étend principalement la méthode du Modèle de Markov Caché Croissant (GHMM) - une variante du modèle probabiliste du Modèle de Markov Caché (HMM) - avec l'application des connaissances préalables pour initialiser une topologie capable de déduire avec précision les " mouvements types " dans la scène. Deuxièmement, le modèle généré se comporte comme une carte auto-organisatrice, apprenant progressivement un comportement piétonnier atypique et le codant dans la topologie tout en mettant à jour les paramètres du HMM sous-jacent.Sur la prédiction, ce travail effectue une inférence bayésienne sur le modèle généré et peut, grâce aux connaissances préalables, réussir à mieux prédire les positions futures des piétons sans disposer de trajectoires de formation, ce qui permet de l'utiliser dans un environnement urbain avec uniquement des données environnementales, que la méthode GHMM actuellement en application.Les contributions de cette thèse sont validées par des résultats expérimentaux sur des données réelles capturées à partir d'une caméra aérienne surplombant une rue urbaine très fréquentée, représentant un environnement bâti structuré et du point de vue de la voiture dans un parking, représentant un environnement semi-structuré et testé sur des trajectoires typiques et atypiques dans chaque cas

Social robot navigation in urban dynamic environments

Deploying mobile robots in social environments requires novel navigation algorithms which are capable of providing valid solutions in such challenging scenarios. The main objective of the present dissertation is to develop new robot navigation approaches able to solve in an intelligent way the navigation problem in urban settings while considering at the same time the interactions with pedestrians, similar to what people easily do with little attention. Before studying in depth navigation algorithms, this thesis focuses on prediction algorithms to provide a more detailed model of the scene. Understanding human motion in outdoor and indoor scenarios is an appealing requirement to characterize correctly urban settings. Urban environments consist essentially of static obstacles and people, which are treated as dynamic and highly uncertain obstacles. Accordingly, it is mandatory to calculate people's intentions in order to successfully build a human prediction model that generates the corresponding human trajectories and considers their interactions with the environment, such as other pedestrians, static obstacles or even robots. It is of great interest that service robots can navigate successfully in typical urban environments, which are dynamic and constrained. In addition, people's behavior should not be conditioned by the presence and the maneuvering of robots. To this end, the robot navigation should seek to minimize its impact on the environment, in our case, on people. This thesis proposes new robot navigation methods that contemplate the social interactions taking place in the scene. In order to procure more intelligence to the navigation algorithm, we propose to integrate seamlessly the human motion prediction information into a new robot planning approach. Real experimentation is essential for the validation of the navigation algorithms. As there are real people involved, we must validate the results in real settings since simulation environments have limitations. In this thesis, we have implemented all the prediction and navigation algorithms in our robotic platform and we have provided plenty of evaluations and testings of our algorithms in real settings.Ubicar robots móviles en entornos sociales requiere novedosos algoritmos de navegación que sean capaces de aportar soluciones válidas en éstos exigentes escenarios. El prinicipal objetivo de la presente disertación es el de desarrollar nuevas soluciones para la navegación de robots que sean capaces de resolver, de una manera más inteligente, los problemas de navegación en emplazamientos urbanos, a la vez que se consideran las interacciones con los transeúntes de manera similar a lo que la gente hace fácilmente prestando poca atención. Antes de estudiar en profundidad los algoritmos de navegación, esta tesis se centra en los algoritmos de predicción para proporcionar un modelo más detallado de la escena. Entender el movimiento humando en entornos exteriores e interiores es un requerimiento deseable para caracterizar correctamente emplazamientos urbanos. Los entornos urbanos están consistituídos por muchos objetos dinámicos y altamente impredecibles, la gente. Por lo tanto, es obligatorio calcular las intenciones de la gente para constriur de manera exitosa un modelo de predicción humano que genere las correspondientes trayectorias humanas y considere sus interacciones con el entorno, como otros peatones, obstáculos estáticos o incluso robots. Es de gran interés que los robots de servicios puedan navegar correctamente en entornos típicamente urbanos, que son dinámicos y acotados, además de que el comportamiento de las personas no debería estar condicionado por la presencia y las maniobras de los robots. Con este fin, la navegación de robots debe buscar minimizar su impacto al entorno, en nuestro caso, a la gente. Esta tesis propone nuevos métodos para la navegación de robots que contemplen las interacciones sociales que suceden en la escena. Para proporcionar una navegación más inteligente, proponemos integrar de manera suave el algoritmo de predicción del movimiento humano con un nuevo enfoque de planificación de trayectorias. La experimentación real es esencial para la validación de los algoritmos de navegación. Ya que hay personas reales implicadas, debemos validar los resultados en emplazamientos reales porque el entorno de simulación tiene limitaciones. En esta tesis hemos implementado todos los algoritmos de predicción y de navegación en la plataforma robótica y hemos proporcionado multitud de evaluaciones y pruebas de nuestros algoritmos en entornos reales

Social robot navigation in urban dynamic environments

A thesis submitted to the Universitat Politècnica de Catalunya for the degree of Doctor of Philosophy, Doctoral Programme: Automatic Control, Robotics and Computer Vision This thesis has been completed at: Institut de Robòtica i Informàtica Industrial, CSIC-UPC.[EN]: Deploying mobile robots in social environments requires novel navigation algorithms which are capable of providing valid solutions in such challenging scenarios. The main objective of the present dissertation is to develop new robot navigation approaches able to solve in an intelligent way the navigation problem in urban settings while considering at the same time the interactions with pedestrians, similar to what people easily do with little attention. Before studying in depth navigation algorithms, this thesis focuses on prediction algorithms to provide a more detailed model of the scene. Understanding human motion in outdoor and indoor scenarios is an appealing requirement to characterize correctly urban settings. Urban environments consist essentially of static obstacles and people, which are treated as dynamic and highly uncertain obstacles. Accordingly, it is mandatory to calculate people's intentions in order to successfully build a human prediction model that generates the corresponding human trajectories and considers their interactions with the environment, such as other pedestrians, static obstacles or even robots. It is of great interest that service robots can navigate successfully in typical urban environments, which are dynamic and constrained. In addition, people's behavior should not be conditioned by the presence and the maneuvering of robots. To this end, the robot navigation should seek to minimize its impact on the environment, in our case, on people. This thesis proposes new robot navigation methods that contemplate the social interactions taking place in the scene. In order to procure more intelligence to the navigation algorithm, we propose to integrate seamlessly the human motion prediction information into a new robot planning approach. Real experimentation is essential for the validation of the navigation algorithms. As there are real people involved, we must validate the results in real settings since simulation environments have limitations. In this thesis, we have implemented all the prediction and navigation algorithms in our robotic platform and we have provided plenty of evaluations and testings of our algorithms in real settings.[ES]: Ubicar robots móviles en entornos sociales requiere novedosos algoritmos de navegación que sean capaces de aportar soluciones válidas en éstos exigentes escenarios. El prinicipal objetivo de la presente disertación es el de desarrollar nuevas soluciones para la navegación de robots que sean capaces de resolver, de una manera más inteligente, los problemas de navegación en emplazamientos urbanos, a la vez que se consideran las interacciones con los transeúntes de manera similar a lo que la gente hace fácilmente prestando poca atención. Antes de estudiar en profundidad los algoritmos de navegación, esta tesis se centra en los algoritmos de predicción para proporcionar un modelo más detallado de la escena. Entender el movimiento humando en entornos exteriores e interiores es un requerimiento deseable para caracterizar correctamente emplazamientos urbanos. Los entornos urbanos están consistituídos por muchos objetos dinámicos y altamente impredecibles, la gente. Por lo tanto, es obligatorio calcular las intenciones de la gente para constriur de manera exitosa un modelo de predicción humano que genere las correspondientes trayectorias humanas y considere sus interacciones con el entorno, como otros peatones, obstáculos estáticos o incluso robots. Es de gran interés que los robots de servicios puedan navegar correctamente en entornos típicamente urbanos, que son dinámicos y acotados, además de que el comportamiento de las personas no debería estar condicionado por la presencia y las maniobras de los robots. Con este fin, la navegación de robots debe buscar minimizar su impacto al entorno, en nuestro caso, a la gente. Esta tesis propone nuevos métodos para la navegación de robots que contemplen las interacciones sociales que suceden en la escena. Para proporcionar una navegación más inteligente, proponemos integrar de manera suave el algoritmo de predicción del movimiento humano con un nuevo enfoque de planificación de trayectorias. La experimentación real es esencial para la validación de los algoritmos de navegación. Ya que hay personas reales implicadas, debemos validar los resultados en emplazamientos reales porque el entorno de simulación tiene limitaciones. En esta tesis hemos implementado todos los algoritmos de predicción y de navegación en la plataforma robótica y hemos proporcionado multitud de evaluaciones y pruebas de nuestros algoritmos en entornos reales.This work has been supported by the research projects: - CSD2007-018 MIPRCV-Consolider: Multimodal, Patter Recognition, and Computer Vision. - DPI2010-17112 RobTaskCoop: Cooperación robots humanos en áreas urbanas - FP7-ICT-2011-7-287617 ARCAS: Aerial Robotics Cooperative Assembly System. - DPI2013-42458-P ROBOT-INT-COOP: Interacción, aprendizaje y cooperación robot-humano en áreas urbanas.Peer Reviewe

Multi-Policy Decision Making for Reliable Navigation in Dynamic Uncertain Environments

Navigating everyday social environments, in the presence of pedestrians and other dynamic obstacles remains one of the key challenges preventing mobile robots from leaving carefully designed spaces and entering our daily lives. The complex and tightly-coupled interactions between these agents make the environment dynamic and unpredictable, posing a formidable problem for robot motion planning. Trajectory planning methods, supported by models of typical human behavior and personal space, often produce reasonable behavior. However, they do not account for the future closed-loop interactions of other agents with the trajectory being constructed. As a consequence, the trajectories are unable to anticipate cooperative interactions (such as a human yielding), or adverse interactions (such as the robot blocking the way). Ideally, the robot must account for coupled agent-agent interactions while reasoning about possible future outcomes, and then take actions to advance towards its navigational goal without inconveniencing nearby pedestrians. Multi-Policy Decision Making (MPDM) is a novel framework for autonomous navigation in dynamic, uncertain environments where the robot's trajectory is not explicitly planned, but instead, the robot dynamically switches between a set of candidate closed-loop policies, allowing it to adapt to different situations encountered in such environments. The candidate policies are evaluated based on short-term (five-second) forward simulations of samples drawn from the estimated distribution of the agents' current states. These forward simulations and thereby the cost function, capture agent-agent interactions as well as agent-robot interactions which depend on the ego-policy being evaluated. In this thesis, we propose MPDM as a new method for navigation amongst pedestrians by dynamically switching from amongst a library of closed-loop policies. Due to real-time constraints, the robot's emergent behavior is directly affected by the quality of policy evaluation. Approximating how good a policy is based on only a few forward roll-outs is difficult, especially with the large space of possible pedestrian configurations and the sensitivity of the forward simulation to the sampled configurations. Traditional methods based on Monte-Carlo sampling often missed likely, high-cost outcomes, resulting in an over-optimistic evaluation of a policy and unreliable emergent behavior. By re-formulating policy evaluation as an optimization problem and enabling the quick discovery of potentially dangerous outcomes, we make MPDM more reliable and risk-aware. Even with the increased reliability, a major limitation is that MPDM requires the system designer to provide a set of carefully hand-crafted policies as it can evaluate only a few policies reliably in real-time. We radically enhance the expressivity of MPDM by allowing policies to have continuous-valued parameters, while simultaneously satisfying real-time constraints by quickly discovering promising policy parameters through a novel iterative gradient-based algorithm. Overall, we reformulate the traditional motion planning problem and paint it in a very different light --- as a bilevel optimization problem where the robot repeatedly discovers likely high-cost outcomes and adapts its policy parameters avoid these outcomes. We demonstrate significant performance benefits through extensive experiments in simulation as well as on a physical robot platform operating in a semi-crowded environment.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/150017/1/dhanvinm_1.pd