Teaching robot’s proactive behavior using human assistance

The final publication is available at link.springer.comIn recent years, there has been a growing interest in enabling autonomous social robots to interact with people. However, many questions remain unresolved regarding the social capabilities robots should have in order to perform this interaction in an ever more natural manner. In this paper, we tackle this problem through a comprehensive study of various topics involved in the interaction between a mobile robot and untrained human volunteers for a variety of tasks. In particular, this work presents a framework that enables the robot to proactively approach people and establish friendly interaction. To this end, we provided the robot with several perception and action skills, such as that of detecting people, planning an approach and communicating the intention to initiate a conversation while expressing an emotional status.We also introduce an interactive learning system that uses the person’s volunteered assistance to incrementally improve the robot’s perception skills. As a proof of concept, we focus on the particular task of online face learning and recognition. We conducted real-life experiments with our Tibi robot to validate the framework during the interaction process. Within this study, several surveys and user studies have been realized to reveal the social acceptability of the robot within the context of different tasks.Peer ReviewedPostprint (author's final draft

Proactive-Cooperative Navigation in Human-Like Environment for Autonomous Robots

This work deals with the problem of navigating a robot in a constrained human-like environment. We provide a method to generate a control strategy that enables the robot to proactively move in order to induce desired and socially acceptable cooperative behaviors in neighboring pedestrians. Contrary to other control strategies that simply aim to passively avoid neighboring pedestrians, this approach aims to simplify the navigation task of a robot by looking for cooperation with humans, especially in crowded and constrained environments. The co-navigation process between humans and a robot is formalized as a multi-objective optimization problem and a control strategy is obtained through the Model Predictive Control (MPC) approach. The Extended Headed Social Force Model with Collision Prediction (EHSFM with CP) is used to predict the human motion. Different social behaviors of humans when moving in a group are also taken into account. A switching strategy between purely reactive and pro active-cooperative planning depending on the evaluation of human intentions is also furnished. Validation of the proactive-cooperative planner enables the robot to generate more socially and understandable behaviors is done with different navigation scenarios

Cooperative social robots: accompanying, guiding and interacting with people

The development of social robots capable of interacting with humans is one of the principal challenges in the field of robotics. More and more, robots are appearing in dynamic environments, like pedestrian walkways, universities, and hospitals; for this reason, their interaction with people must be conducted in a natural, gradual, and cordial manner, given that their function could be aid, or assist people. Therefore, navigation and interaction among humans in these environments are key skills that future generations of robots will require to have. Additionally, robots must also be able to cooperate with each other, if necessary. This dissertation examines these various challenges and describes the development of a set of techniques that allow robots to interact naturally with people in their environments, as they guide or accompany humans in urban zones. In this sense, the robots' movements are inspired by the persons' actions and gestures, determination of appropriate personal space, and the rules of common social convention. The first issue this thesis tackles is the development of an innovative robot-companion approach based on the newly founded Extended Social-Forces Model. We evaluate how people navigate and we formulate a set of virtual social forces to describe robot's behavior in terms of motion. Moreover, we introduce a robot companion analytical metric to effectively evaluate the system. This assessment is based on the notion of "proxemics" and ensures that the robot's navigation is socially acceptable by the person being accompanied, as well as to other pedestrians in the vicinity. Through a user study, we show that people interpret the robot's behavior according to human social norms. In addition, a new framework for guiding people in urban areas with a set of cooperative mobile robots is presented. The proposed approach offers several significant advantages, as compared with those outlined in prior studies. Firstly, it allows a group of people to be guided within both open and closed areas; secondly, it uses several cooperative robots; and thirdly, it includes features that enable the robots to keep people from leaving the crowd group, by approaching them in a friendly and safe manner. At the core of our approach, we propose a "Discrete Time Motion" model, which works to represent human and robot motions, to predict people's movements, so as to plan a route and provide the robots with concrete motion instructions. After, this thesis goes one step forward by developing the "Prediction and Anticipation Model". This model enables us to determine the optimal distribution of robots for preventing people from straying from the formation in specific areas of the map, and thus to facilitate the task of the robots. Furthermore, we locally optimize the work performed by robots and people alike, and thereby yielding a more human-friendly motion. Finally, an autonomous mobile robot capable of interacting to acquire human-assisted learning is introduced. First, we present different robot behaviors to approach a person and successfully engage with him/her. On the basis of this insight, we furnish our robot with a simple visual module for detecting human faces in real-time. We observe that people ascribe different personalities to the robot depending on its different behaviors. Once contact is initiated, people are given the opportunity to assist the robot to improve its visual skills. After this assisted learning stage, the robot is able to detect people by using the enhanced visual methods. Both contributions are extensively and rigorously tested in real environments. As a whole, this thesis demonstrates the need for robots that are able to operate acceptably around people; to behave in accordance with social norms while accompanying and guiding them. Furthermore, this work shows that cooperation amongst a group of robots optimizes the performance of the robots and people as well.El desenvolupament de robots socials capaços d'interactuar amb els éssers humans és un dels principals reptes en el camp de la robòtica. Actualment, els robots comencen a aparèixer en entorns dinàmics, com zones de vianants, universitats o hospitals; per aquest motiu, aquesta interacció ha de realitzar-se de manera natural, progressiva i cordial, ja que la seva utilització pot ser col.laboració, assistència o ajuda a les persones. Per tant, la navegació i la interacció amb els humans, en aquests entorns, són habilitats importants que les futures generacions de robots han de posseir, a més a més, els robots han de ser aptes de cooperar entre ells si fos requerit. El present treball estudia aquests reptes plantejats. S’han desenvolupat un conjunt de tècniques que permeten als robots interectuar de manera natural amb les persones i el seu entorn, mentre que guien o acompanyen als humans en zones urbanes. En aquest sentit, el moviment dels robots s’inspira en la manera com es mouen els humans en les convenvions socials, així com l’espai personal.El primer punt que aquesta tesi comprèn és el desenvolupament d’un nou mètode per a "robots-acompanyants" basat en el nou model estès de forces socials. S’ha evaluat com es mouen les persones i s’han formulat un conjunt de forces socials virtuals que descriuren el comportament del robot en termes de moviments. Aquesta evaluació es basa en el concepte de “proxemics” i assegura que la navegació del robot està socialment acceptada per la persona que està sent acompanyada i per la gent que es troba a l’entorn. Per mitjà d’un estudi social, mostrem que els humans interpreten el comportament del robot d’acord amb les normes socials. Així mateix, un nou sistema per a guiar a persones en zones urbanes amb un conjunt de robots mòbils que cooperen és presentat. El model proposat ofereix diferents avantatges comparat amb treballs anteriors. Primer, es permet a un grup de persones ser guiades en entorns oberts o amb alta densitat d’obstacles; segon, s’utilitzen diferents robots que cooperen; tercer, els robots són capaços de reincorporar a la formació les persones que s’han allunyat del grup anteriorment de manera segura. La base del nostre enfocament es basa en el nou model anomenat “Discrete Time Motion”, el qual representa els movimients dels humans i els robots, prediu el comportament de les persones, i planeja i proporciona una ruta als robots.Posteriorment, aquesta tesi va un pas més enllà amb el desenvolupament del model “Prediction and Anticipation Model”. Aquest model ens permet determinar la distribució òptima de robots per a prevenir que les persones s’allunyin del grup en zones especíıfiques del mapa, i per tant facilitar la tasca dels robots. A més, s’optimitza localment el treball realitzat pels robots i les persones, produint d’aquesta manera un moviment més amigable. Finalment, s’introdueix un robot autònom mòbil capaç d’interactuar amb les persones per realitzar un aprenentatge assistit. Incialment, es presenten diferents comportaments del robot per apropar-se a una persona i crear un víıncle amb ell/ella. Basant-nos en aquesta idea, un mòdul visual per a la detecció de cares humanes en temps real va ser proporcionat al robot. Hem observat que les persones atribueixen diferents personalitats al robot en funció dels seus diferents comportaments. Una vegada que el contacte va ser iniciat es va donar l’oportunitat als voluntaris d’ajudar al robot per a millorar les seves habilitats visuals. Després d’aquesta etapa d’aprenentatge assistit, el robot va ser capaç d’identificar a les persones mitjançant l'ús de mètodes visuals.En resum, aquesta tesi presenta i demostra la necessitat de robots que siguin capaços d’operar de forma acceptable amb la gent i que es comportin d’acord amb les normes socials mentres acompanyen o guien a persones. Per altra banda, aquest treball mostra que la coperació entre un grup de robots pot optimitzar el rendiment tant dels robots com dels humans

Human-aware space sharing and navigation for an interactive robot

Les méthodes de planification de mouvements robotiques se sont développées à un rythme accéléré ces dernières années. L'accent a principalement été mis sur le fait de rendre les robots plus efficaces, plus sécurisés et plus rapides à réagir à des situations imprévisibles. En conséquence, nous assistons de plus en plus à l'introduction des robots de service dans notre vie quotidienne, en particulier dans les lieux publics tels que les musées, les centres commerciaux et les aéroports. Tandis qu'un robot de service mobile se déplace dans l'environnement humain, il est important de prendre en compte l'effet de son comportement sur les personnes qu'il croise ou avec lesquelles il interagit. Nous ne les voyons pas comme de simples machines, mais comme des agents sociaux et nous nous attendons à ce qu'ils se comportent de manière similaire à l'homme en suivant les normes sociétales comme des règles. Ceci a créé de nouveaux défis et a ouvert de nouvelles directions de recherche pour concevoir des algorithmes de commande de robot, qui fournissent des comportements de robot acceptables, lisibles et proactifs. Cette thèse propose une méthode coopérative basée sur l'optimisation pour la planification de trajectoire et la navigation du robot avec des contraintes sociales intégrées pour assurer des mouvements de robots prudents, conscients de la présence de l'être humain et prévisibles. La trajectoire du robot est ajustée dynamiquement et continuellement pour satisfaire ces contraintes sociales. Pour ce faire, nous traitons la trajectoire du robot comme une bande élastique (une construction mathématique représentant la trajectoire du robot comme une série de positions et une différence de temps entre ces positions) qui peut être déformée (dans l'espace et dans le temps) par le processus d'optimisation pour respecter les contraintes données. De plus, le robot prédit aussi les trajectoires humaines plausibles dans la même zone d'exploitation en traitant les chemins humains aussi comme des bandes élastiques. Ce système nous permet d'optimiser les trajectoires des robots non seulement pour le moment présent, mais aussi pour l'interaction entière qui se produit lorsque les humains et les robots se croisent les uns les autres. Nous avons réalisé un ensemble d'expériences avec des situations interactives humains-robots qui se produisent dans la vie de tous les jours telles que traverser un couloir, passer par une porte et se croiser sur de grands espaces ouverts. La méthode de planification coopérative proposée se compare favorablement à d'autres schémas de planification de la navigation à la pointe de la technique. Nous avons augmenté le comportement de navigation du robot avec un mouvement synchronisé et réactif de sa tête. Cela permet au robot de regarder où il va et occasionnellement de détourner son regard vers les personnes voisines pour montrer que le robot va éviter toute collision possible avec eux comme prévu par le planificateur. À tout moment, le robot pondère les multiples critères selon le contexte social et décide de ce vers quoi il devrait porter le regard. Grâce à une étude utilisateur en ligne, nous avons montré que ce mécanisme de regard complète efficacement le comportement de navigation ce qui améliore la lisibilité des actions du robot. Enfin, nous avons intégré notre schéma de navigation avec un système de supervision plus large qui peut générer conjointement des comportements du robot standard tel que l'approche d'une personne et l'adaptation de la vitesse du robot selon le groupe de personnes que le robot guide dans des scénarios d'aéroport ou de musée.The methods of robotic movement planning have grown at an accelerated pace in recent years. The emphasis has mainly been on making robots more efficient, safer and react faster to unpredictable situations. As a result we are witnessing more and more service robots introduced in our everyday lives, especially in public places such as museums, shopping malls and airports. While a mobile service robot moves in a human environment, it leaves an innate effect on people about its demeanor. We do not see them as mere machines but as social agents and expect them to behave humanly by following societal norms and rules. This has created new challenges and opened new research avenues for designing robot control algorithms that deliver human-acceptable, legible and proactive robot behaviors. This thesis proposes a optimization-based cooperative method for trajectoryplanning and navigation with in-built social constraints for keeping robot motions safe, human-aware and predictable. The robot trajectory is dynamically and continuously adjusted to satisfy these social constraints. To do so, we treat the robot trajectory as an elastic band (a mathematical construct representing the robot path as a series of poses and time-difference between those poses) which can be deformed (both in space and time) by the optimization process to respect given constraints. Moreover, we also predict plausible human trajectories in the same operating area by treating human paths also as elastic bands. This scheme allows us to optimize the robot trajectories not only for the current moment but for the entire interaction that happens when humans and robot cross each other's paths. We carried out a set of experiments with canonical human-robot interactive situations that happen in our everyday lives such as crossing a hallway, passing through a door and intersecting paths on wide open spaces. The proposed cooperative planning method compares favorably against other stat-of-the-art human-aware navigation planning schemes. We have augmented robot navigation behavior with synchronized and responsive movements of the robot head, making the robot look where it is going and occasionally diverting its gaze towards nearby people to acknowledge that robot will avoid any possible collision with them. At any given moment the robot weighs multiple criteria according to the social context and decides where it should turn its gaze. Through an online user study we have shown that such gazing mechanism effectively complements the navigation behavior and it improves legibility of the robot actions. Finally, we have integrated our navigation scheme with a broader supervision system which can jointly generate normative robot behaviors such as approaching a person and adapting the robot speed according to a group of people who the robot guides in airports or museums

Combining motion planning with social reward sources for collaborative human-robot navigation task design

Across the human history, teamwork is one of the main pillars sustaining civilizations and technology development. In consequence, as the world embraces omatization, human-robot collaboration arises naturally as a cornerstone. This applies to a huge spectrum of tasks, most of them involving navigation. As a result, tackling pure collaborative navigation tasks can be a good first foothold for roboticists in this enterprise. In this thesis, we define a useful framework for knowledge representation in human-robot collaborative navigation tasks and propose a first solution to the human-robot collaborative search task. After validating the model, two derived projects tackling its main weakness are introduced: the compilation of a human search dataset and the implementation of a multi-agent planner for human-robot navigatio

Look Both Ways: Intersections Of Past And Present In The Shaping Of Relations Between Cyclists, Pedestrians, And Driverless Cars

Driverless cars are expected to transform society in many ways. Since nowadays most collisions are due to human error, safety is among the most anticipated benefits of the technology. The promise of near zero fatalities on roads appears in many industry statements and government reports. Because of that, every collision, especially involving fatalities, receives much attention from the media and public. That kind of scrutiny resembles the early days of the conventional automobiles. In those days, automobiles – also called “horseless carriages” – were not well received by the majority of the population. Cars brought conflicts and fatalities on roads to a level never seen before. The automobile industry, using public relations, shifted society’s perception about who belongs to the roads, and who should be blamed for the rise of fatalities. That shift influenced legislation and tort law in motor-vehicle centric ways. It also created cities with infrastructure focused on the automobile at the expense of other means of transportation. Today, one of the most difficult challenges for driverless cars is the unpredictability of pedestrian and cyclist behaviour. To accelerate the deployment of the technology, some are considering the necessity of law enforcement against pedestrians and other street users. Centred on urban environments, pedestrians and cyclists, and with an interdisciplinary and advocacy-oriented approach, this thesis seeks to contribute to the debate about the safety and deployment of driverless cars, its influence on law and legislation, and how a car-centred view of the technology may limit its potentialities

Proactive-Cooperative Navigation in Human-Like Environment for Autonomous Robots

International audienceThis work d deals with the problem of navigating a robot in a constrained human-like environment. We provide a method to generate a control strategy that enables the robot to proactively move in order to induce desired and socially acceptable cooperative behaviors in neighboring pedestrians. Contrary to other control strategies that simply aim to passively avoid neighboring pedestrians, this approach aims to simplify the navigation task of a robot by looking for cooperation with humans, especially in crowded and constrained environments. The co-navigation process between humans and a robot is formalized as a multi-objective optimization problem and a control strategy is obtained through the Model Predictive Control (MPC) approach. The Extended Headed Social Force Model with Collision Prediction (EHSFM with CP) is used to predict the human motion. Different social behaviors of humans when moving in a group are also taken into account. A switching strategy between purely reactive and proactive-cooperative planning depending on the evaluation of human intentions is also furnished. Validation of the proactive-cooperative planner enables the robot to generate more socially and understandable behaviors is done with different navigation scenarios

Environment and task modeling of long-term-autonomous service robots

Utilizing service robots in real-world tasks can significantly improve efficiency, productivity, and safety in various fields such as healthcare, hospitality, and transportation. However, integrating these robots into complex, human-populated environments for continuous use is a significant challenge. A key potential for addressing this challenge lies in long-term modeling capabilities to navigate, understand, and proactively exploit these environments for increased safety and better task performance. For example, robots may use this long-term knowledge of human activity to avoid crowded spaces when navigating or improve their human-centric services. This thesis proposes comprehensive approaches to improve the mapping, localization, and task fulfillment capabilities of service robots by leveraging multi-modal sensor information and (long- term) environment modeling. Learned environmental dynamics are actively exploited to improve the task performance of service robots. As a first contribution, a new long-term-autonomous service robot is presented, designed for both inside and outside buildings. The multi-modal sensor information provided by the robot forms the basis for subsequent methods to model human-centric environments and human activity. It is shown that utilizing multi-modal data for localization and mapping improves long-term robustness and map quality. This especially applies to environments of varying types, i.e., mixed indoor and outdoor or small-scale and large-scale areas. Another essential contribution is a regression model for spatio-temporal prediction of human activity. The model is based on long-term observations of humans by a mobile robot. It is demonstrated that the proposed model can effectively represent the distribution of detected people resulting from moving robots and enables proactive navigation planning. Such model predictions are then used to adapt the robot’s behavior by synthesizing a modular task control model. A reactive executive system based on behavior trees is introduced, which actively triggers recovery behaviors in the event of faults to improve the long-term autonomy. By explicitly addressing failures of robot software components and more advanced problems, it is shown that errors can be solved and potential human helpers can be found efficiently

Evaluating the use of human aware navigation in industrial robot arms

Although the principles followed by modern standards for interaction between humans and robots follow the First Law of Robotics popularized in science fiction in the 1960s, the current standards regulating the interaction between humans and robots emphasize the importance of physical safety. However, they are less developed in another key dimension: psychological safety. As sales of industrial robots have been increasing over recent years, so has the frequency of human–robot interaction (HRI). The present article looks at the current safety guidelines for HRI in an industrial setting and assesses their suitability. This article then presents a means to improve current standards utilizing lessons learned from studies into human aware navigation (HAN), which has seen increasing use in mobile robotics. This article highlights limitations in current research, where the relationships established in mobile robotics have not been carried over to industrial robot arms. To understand this, it is necessary to focus less on how a robot arm avoids humans and more on how humans react when a robot is within the same space. Currently, the safety guidelines are behind the technological advance, however, with further studies aimed at understanding HRI and applying it to newly developed path finding and obstacle avoidance methods, science fiction can become science fact