Human-aware space sharing and navigation for an interactive robot

Les méthodes de planification de mouvements robotiques se sont développées à un rythme accéléré ces dernières années. L'accent a principalement été mis sur le fait de rendre les robots plus efficaces, plus sécurisés et plus rapides à réagir à des situations imprévisibles. En conséquence, nous assistons de plus en plus à l'introduction des robots de service dans notre vie quotidienne, en particulier dans les lieux publics tels que les musées, les centres commerciaux et les aéroports. Tandis qu'un robot de service mobile se déplace dans l'environnement humain, il est important de prendre en compte l'effet de son comportement sur les personnes qu'il croise ou avec lesquelles il interagit. Nous ne les voyons pas comme de simples machines, mais comme des agents sociaux et nous nous attendons à ce qu'ils se comportent de manière similaire à l'homme en suivant les normes sociétales comme des règles. Ceci a créé de nouveaux défis et a ouvert de nouvelles directions de recherche pour concevoir des algorithmes de commande de robot, qui fournissent des comportements de robot acceptables, lisibles et proactifs. Cette thèse propose une méthode coopérative basée sur l'optimisation pour la planification de trajectoire et la navigation du robot avec des contraintes sociales intégrées pour assurer des mouvements de robots prudents, conscients de la présence de l'être humain et prévisibles. La trajectoire du robot est ajustée dynamiquement et continuellement pour satisfaire ces contraintes sociales. Pour ce faire, nous traitons la trajectoire du robot comme une bande élastique (une construction mathématique représentant la trajectoire du robot comme une série de positions et une différence de temps entre ces positions) qui peut être déformée (dans l'espace et dans le temps) par le processus d'optimisation pour respecter les contraintes données. De plus, le robot prédit aussi les trajectoires humaines plausibles dans la même zone d'exploitation en traitant les chemins humains aussi comme des bandes élastiques. Ce système nous permet d'optimiser les trajectoires des robots non seulement pour le moment présent, mais aussi pour l'interaction entière qui se produit lorsque les humains et les robots se croisent les uns les autres. Nous avons réalisé un ensemble d'expériences avec des situations interactives humains-robots qui se produisent dans la vie de tous les jours telles que traverser un couloir, passer par une porte et se croiser sur de grands espaces ouverts. La méthode de planification coopérative proposée se compare favorablement à d'autres schémas de planification de la navigation à la pointe de la technique. Nous avons augmenté le comportement de navigation du robot avec un mouvement synchronisé et réactif de sa tête. Cela permet au robot de regarder où il va et occasionnellement de détourner son regard vers les personnes voisines pour montrer que le robot va éviter toute collision possible avec eux comme prévu par le planificateur. À tout moment, le robot pondère les multiples critères selon le contexte social et décide de ce vers quoi il devrait porter le regard. Grâce à une étude utilisateur en ligne, nous avons montré que ce mécanisme de regard complète efficacement le comportement de navigation ce qui améliore la lisibilité des actions du robot. Enfin, nous avons intégré notre schéma de navigation avec un système de supervision plus large qui peut générer conjointement des comportements du robot standard tel que l'approche d'une personne et l'adaptation de la vitesse du robot selon le groupe de personnes que le robot guide dans des scénarios d'aéroport ou de musée.The methods of robotic movement planning have grown at an accelerated pace in recent years. The emphasis has mainly been on making robots more efficient, safer and react faster to unpredictable situations. As a result we are witnessing more and more service robots introduced in our everyday lives, especially in public places such as museums, shopping malls and airports. While a mobile service robot moves in a human environment, it leaves an innate effect on people about its demeanor. We do not see them as mere machines but as social agents and expect them to behave humanly by following societal norms and rules. This has created new challenges and opened new research avenues for designing robot control algorithms that deliver human-acceptable, legible and proactive robot behaviors. This thesis proposes a optimization-based cooperative method for trajectoryplanning and navigation with in-built social constraints for keeping robot motions safe, human-aware and predictable. The robot trajectory is dynamically and continuously adjusted to satisfy these social constraints. To do so, we treat the robot trajectory as an elastic band (a mathematical construct representing the robot path as a series of poses and time-difference between those poses) which can be deformed (both in space and time) by the optimization process to respect given constraints. Moreover, we also predict plausible human trajectories in the same operating area by treating human paths also as elastic bands. This scheme allows us to optimize the robot trajectories not only for the current moment but for the entire interaction that happens when humans and robot cross each other's paths. We carried out a set of experiments with canonical human-robot interactive situations that happen in our everyday lives such as crossing a hallway, passing through a door and intersecting paths on wide open spaces. The proposed cooperative planning method compares favorably against other stat-of-the-art human-aware navigation planning schemes. We have augmented robot navigation behavior with synchronized and responsive movements of the robot head, making the robot look where it is going and occasionally diverting its gaze towards nearby people to acknowledge that robot will avoid any possible collision with them. At any given moment the robot weighs multiple criteria according to the social context and decides where it should turn its gaze. Through an online user study we have shown that such gazing mechanism effectively complements the navigation behavior and it improves legibility of the robot actions. Finally, we have integrated our navigation scheme with a broader supervision system which can jointly generate normative robot behaviors such as approaching a person and adapting the robot speed according to a group of people who the robot guides in airports or museums

Social Perception of Pedestrians and Virtual Agents Using Movement Features

In many tasks such as navigation in a shared space, humans explicitly or implicitly estimate social information related to the emotions, dominance, and friendliness of other humans around them. This social perception is critical in predicting others’ motions or actions and deciding how to interact with them. Therefore, modeling social perception is an important problem for robotics, autonomous vehicle navigation, and VR and AR applications. In this thesis, we present novel, data-driven models for the social perception of pedestrians and virtual agents based on their movement cues, including gaits, gestures, gazing, and trajectories. We use deep learning techniques (e.g., LSTMs) along with biomechanics to compute the gait features and combine them with local motion models to compute the trajectory features. Furthermore, we compute the gesture and gaze representations using psychological characteristics. We describe novel mappings between these computed gaits, gestures, gazing, and trajectory features and the various components (emotions, dominance, friendliness, approachability, and deception) of social perception. Our resulting data-driven models can identify the dominance, deception, and emotion of pedestrians from videos with an accuracy of more than 80%. We also release new datasets to evaluate these methods. We apply our data-driven models to socially-aware robot navigation and the navigation of autonomous vehicles among pedestrians. Our method generates robot movement based on pedestrians’ dominance levels, resulting in higher rapport and comfort. We also apply our data-driven models to simulate virtual agents with desired emotions, dominance, and friendliness. We perform user studies and show that our data-driven models significantly increase the user’s sense of social presence in VR and AR environments compared to the baseline methods.Doctor of Philosoph

Interactive Motion Planning for Multi-agent Systems with Physics-based and Behavior Constraints

Man-made entities and humans rely on movement as an essential form of interaction with the world. Whether it is an autonomous vehicle navigating crowded roadways or a simulated pedestrian traversing a virtual world, each entity must compute safe, effective paths to achieve their goals. In addition, these entities, termed agents, are subject to unique physical and behavioral limitations within their environment. For example, vehicles have a finite physical turning radius and must obey behavioral constraints such as traffic signals and rules of the road. Effective motion planning algorithms for diverse agents must account for these physics-based and behavior constraints. In this dissertation, we present novel motion planning algorithms that account for constraints which physically limit the agent and impose behavioral limitations on the virtual agents. We describe representational approaches to capture specific physical constraints on the various agents and propose abstractions to model behavior constraints affecting them. We then describe algorithms to plan motions for agents who are subject to the modeled constraints. First, we describe a biomechanically accurate elliptical representation for virtual pedestrians; we also describe human-like movement constraints corresponding to shoulder-turning and side-stepping in dense environments. We detail a novel motion planning algorithm extending velocity obstacles to generate collisionfree paths for hundreds of elliptical agents at interactive rates. Next, we describe an algorithm to encode dynamics and traffic-like behavior constraints for autonomous vehicles in urban and highway environments. We describe a motion planning algorithm to generate safe, high-speed avoidance maneuvers using a novel optimization function and modified control obstacle formulation, and we also present a simulation framework to evaluate driving strategies. Next, we present an approach to incorporate high-level reasoning to model the motions and behaviors of virtual agents in terms of verbal interactions with other agents or avatars. Our approach leverages natural-language interaction to reduce uncertainty and generate effective plans. Finally, we describe an application of our techniques to simulate pedestrian behaviors for gathering simulated data about loading, unloading, and evacuating an aircraft.Doctor of Philosoph

Social robot navigation in urban dynamic environments

Deploying mobile robots in social environments requires novel navigation algorithms which are capable of providing valid solutions in such challenging scenarios. The main objective of the present dissertation is to develop new robot navigation approaches able to solve in an intelligent way the navigation problem in urban settings while considering at the same time the interactions with pedestrians, similar to what people easily do with little attention. Before studying in depth navigation algorithms, this thesis focuses on prediction algorithms to provide a more detailed model of the scene. Understanding human motion in outdoor and indoor scenarios is an appealing requirement to characterize correctly urban settings. Urban environments consist essentially of static obstacles and people, which are treated as dynamic and highly uncertain obstacles. Accordingly, it is mandatory to calculate people's intentions in order to successfully build a human prediction model that generates the corresponding human trajectories and considers their interactions with the environment, such as other pedestrians, static obstacles or even robots. It is of great interest that service robots can navigate successfully in typical urban environments, which are dynamic and constrained. In addition, people's behavior should not be conditioned by the presence and the maneuvering of robots. To this end, the robot navigation should seek to minimize its impact on the environment, in our case, on people. This thesis proposes new robot navigation methods that contemplate the social interactions taking place in the scene. In order to procure more intelligence to the navigation algorithm, we propose to integrate seamlessly the human motion prediction information into a new robot planning approach. Real experimentation is essential for the validation of the navigation algorithms. As there are real people involved, we must validate the results in real settings since simulation environments have limitations. In this thesis, we have implemented all the prediction and navigation algorithms in our robotic platform and we have provided plenty of evaluations and testings of our algorithms in real settings.Ubicar robots móviles en entornos sociales requiere novedosos algoritmos de navegación que sean capaces de aportar soluciones válidas en éstos exigentes escenarios. El prinicipal objetivo de la presente disertación es el de desarrollar nuevas soluciones para la navegación de robots que sean capaces de resolver, de una manera más inteligente, los problemas de navegación en emplazamientos urbanos, a la vez que se consideran las interacciones con los transeúntes de manera similar a lo que la gente hace fácilmente prestando poca atención. Antes de estudiar en profundidad los algoritmos de navegación, esta tesis se centra en los algoritmos de predicción para proporcionar un modelo más detallado de la escena. Entender el movimiento humando en entornos exteriores e interiores es un requerimiento deseable para caracterizar correctamente emplazamientos urbanos. Los entornos urbanos están consistituídos por muchos objetos dinámicos y altamente impredecibles, la gente. Por lo tanto, es obligatorio calcular las intenciones de la gente para constriur de manera exitosa un modelo de predicción humano que genere las correspondientes trayectorias humanas y considere sus interacciones con el entorno, como otros peatones, obstáculos estáticos o incluso robots. Es de gran interés que los robots de servicios puedan navegar correctamente en entornos típicamente urbanos, que son dinámicos y acotados, además de que el comportamiento de las personas no debería estar condicionado por la presencia y las maniobras de los robots. Con este fin, la navegación de robots debe buscar minimizar su impacto al entorno, en nuestro caso, a la gente. Esta tesis propone nuevos métodos para la navegación de robots que contemplen las interacciones sociales que suceden en la escena. Para proporcionar una navegación más inteligente, proponemos integrar de manera suave el algoritmo de predicción del movimiento humano con un nuevo enfoque de planificación de trayectorias. La experimentación real es esencial para la validación de los algoritmos de navegación. Ya que hay personas reales implicadas, debemos validar los resultados en emplazamientos reales porque el entorno de simulación tiene limitaciones. En esta tesis hemos implementado todos los algoritmos de predicción y de navegación en la plataforma robótica y hemos proporcionado multitud de evaluaciones y pruebas de nuestros algoritmos en entornos reales