Designing a robot to evaluate group formations

Robots are making their way in environments inhabited by people. Whether in domestic or public crowded environments, robots should take into consideration social norms and behaviors in order to become a social robot. This dissertation focuses on the problem of how to develop a robotic platform in order to validate human-robot interaction experiences in realistic environments. More specifically, we are concerned with social interactions in human-robot groups in public scenarios, where a variety of people can converge. Our final goal is the develop of a social robot based on certain theories of group behavior and the use of space, known as spatial relationships. The intermediate goals are related with the design and development of the experiences in the wild: as minor changes as possible in the scenario, definition of social tasks, gradual development of a robotic platform as transparent as possible from the robotic side. Initially, this research introduces several preliminary studies of human-robot interaction (HRI) with the PAL Robotics’ REEM robot at the CosmoCaixa Science Museum in Barcelona. Based on certain theories about the use of space as a form of social communication or interaction, the task under study with the commercial robot is as a museum guide, both when the group was in motion (\ie when it was being guided) as well as a group in a static place. Moreover, a second HRI study with REEM robot accomplishing the task of a teacher's assistant was carried out to analyze the perception of the robot's social presence and identity. Likewise, the development of a robotic platform, known as MASHI, for the study of HRI is presented. Based on the service to be completed by the robot, improvements in the experimental robotic platform (structure, morphology, head, face, arms) were carried out in continuous cycles following the development of HRI experiences. This structure should be hold as simple as possible in order to make it `transparent' in the social HRI study. Next, the field study of human-robot social interaction with the MASHI robot with the role of exhibition guide in a cultural center is presented. Based on direct observation techniques, a study is made of the different spatial relationships that are generated when a robot interacts with a person or groups of people. Finally, a novel approach to represent the spatial relationships of HRI in a qualitative way is introduced for future experiences. In this concluding study, we analyze different spatial arrangements generated in a social scenario with a robot within the guide role. As a main conclusion, it can be stated that people follow social norms, in the form of spatial relationships, when interacting with a robot that provide a social service in a public space. Children, however, recurrently challenge these social norms, probably because they are constantly learning about the norms that regulate our coexistence. Spatial relationships are clearly reinforced when the role assigned to the robot is more explicit and understood by people. Spatial relationships can be affected by the characteristics of the environment, either by the available space or by the elements arranged in it, as well as by the number of people who inhabit it. Overall, this dissertation points out that the provided service, and its understanding from the user’s side, is more important that the robotic skills of the robotic platform in order to improve user experiences in public environments.Los robots se abren paso en entornos habitados por personas. Ya sea en entornos domésticos o públicos, los robots deben tener en cuenta ciertas normas y comportamientos sociales para convertirse en un robot social. Esta disertación se centra en el problema de cómo desarrollar una plataforma robótica para validar experiencias de interacción humano-robot en entornos realistas. Más específicamente, nos preocupamos por las interacciones sociales en grupos humano-robot en escenarios públicos, donde una gran variedad de personas puede converger. Nuestro objetivo final es el desarrollo de un robot social basado en ciertas teorías de comportamiento grupal y el uso del espacio, conocidas como relaciones espaciales. Los objetivos intermedios están relacionados con el diseño y desarrollo de las experiencias `en la naturaleza': cambios mínimos como sea posible en el escenario, definición de tareas sociales, desarrollo gradual de una plataforma robótica lo más transparente posible desde el lado robótico. Inicialmente, esta investigación presenta varios estudios preliminares de interacción humano-robot (HRI) con el robot REEM de PAL Robotics en el Museo de Ciencias CosmoCaixa de Barcelona. Basado en ciertas teorías sobre el uso del espacio como una forma de comunicación o interacción social, la tarea en este estudio con el robot comercial es como guía de museo, tanto cuando el grupo estaba en movimiento (es decir, cuando estaba siendo guiado) como cuando el grupo estaba en un lugar estático. Además, se llevó a cabo un segundo estudio de HRI con un robot REEM que realizaba la tarea de un asistente de profesor para analizar la percepción de la presencia e identidad social del robot. Asimismo, se presenta el desarrollo de una plataforma robótica, conocida como MASHI, para el estudio de la HRI. En función del servicio que debe completar el robot, las mejoras en la plataforma robótica experimental (estructura, morfología, cabeza, cara, brazos) se llevaron a cabo en ciclos continuos siguiendo el desarrollo de las experiencias de HRI. Esta estructura debe mantenerse lo más simple posible para que sea 'transparente' en el estudio de HRI social. A continuación, se presenta el estudio de campo de la interacción social humano-robot con el robot MASHI con el papel de guía de exposición en un centro cultural. Con base en técnicas de observación directa, se realiza un estudio de las diferentes relaciones espaciales que se generan cuando un robot interactúa con una persona o grupos de personas. Finalmente, se introduce un enfoque novedoso para representar las relaciones espaciales de la HRI de forma cualitativa para las experiencias futuras. En este estudio final, analizamos diferentes arreglos espaciales generados en un escenario social con un robot con el rol de guía. Como conclusión principal, se puede afirmar que las personas siguen normas sociales, en forma de relaciones espaciales, cuando interactúan con un robot que brinda un servicio social en un espacio público. Los niños, sin embargo, desafían recurrentemente estas normas sociales, probablemente porque están aprendiendo constantemente sobre las normas que regulan nuestra convivencia. Las relaciones espaciales se refuerzan claramente cuando el rol asignado al robot es más explícito y entendido por las personas. Las relaciones espaciales pueden verse afectadas por las características del entorno, ya sea por el espacio disponible o por los elementos dispuestos en él, así como por el número de personas que lo habitan. En general, esta disertación señala que el servicio prestado, y su comprensión del lado del usuario, es más importante que las habilidades robóticas de la plataforma robótica con el fin de mejorar las experiencias del usuario en entornos público

Designing a robot to evaluate group formations

Robots are making their way in environments inhabited by people. Whether in domestic or public crowded environments, robots should take into consideration social norms and behaviors in order to become a social robot. This dissertation focuses on the problem of how to develop a robotic platform in order to validate human-robot interaction experiences in realistic environments. More specifically, we are concerned with social interactions in human-robot groups in public scenarios, where a variety of people can converge. Our final goal is the develop of a social robot based on certain theories of group behavior and the use of space, known as spatial relationships. The intermediate goals are related with the design and development of the experiences in the wild: as minor changes as possible in the scenario, definition of social tasks, gradual development of a robotic platform as transparent as possible from the robotic side. Initially, this research introduces several preliminary studies of human-robot interaction (HRI) with the PAL Robotics’ REEM robot at the CosmoCaixa Science Museum in Barcelona. Based on certain theories about the use of space as a form of social communication or interaction, the task under study with the commercial robot is as a museum guide, both when the group was in motion (\ie when it was being guided) as well as a group in a static place. Moreover, a second HRI study with REEM robot accomplishing the task of a teacher's assistant was carried out to analyze the perception of the robot's social presence and identity. Likewise, the development of a robotic platform, known as MASHI, for the study of HRI is presented. Based on the service to be completed by the robot, improvements in the experimental robotic platform (structure, morphology, head, face, arms) were carried out in continuous cycles following the development of HRI experiences. This structure should be hold as simple as possible in order to make it `transparent' in the social HRI study. Next, the field study of human-robot social interaction with the MASHI robot with the role of exhibition guide in a cultural center is presented. Based on direct observation techniques, a study is made of the different spatial relationships that are generated when a robot interacts with a person or groups of people. Finally, a novel approach to represent the spatial relationships of HRI in a qualitative way is introduced for future experiences. In this concluding study, we analyze different spatial arrangements generated in a social scenario with a robot within the guide role. As a main conclusion, it can be stated that people follow social norms, in the form of spatial relationships, when interacting with a robot that provide a social service in a public space. Children, however, recurrently challenge these social norms, probably because they are constantly learning about the norms that regulate our coexistence. Spatial relationships are clearly reinforced when the role assigned to the robot is more explicit and understood by people. Spatial relationships can be affected by the characteristics of the environment, either by the available space or by the elements arranged in it, as well as by the number of people who inhabit it. Overall, this dissertation points out that the provided service, and its understanding from the user’s side, is more important that the robotic skills of the robotic platform in order to improve user experiences in public environments.Los robots se abren paso en entornos habitados por personas. Ya sea en entornos domésticos o públicos, los robots deben tener en cuenta ciertas normas y comportamientos sociales para convertirse en un robot social. Esta disertación se centra en el problema de cómo desarrollar una plataforma robótica para validar experiencias de interacción humano-robot en entornos realistas. Más específicamente, nos preocupamos por las interacciones sociales en grupos humano-robot en escenarios públicos, donde una gran variedad de personas puede converger. Nuestro objetivo final es el desarrollo de un robot social basado en ciertas teorías de comportamiento grupal y el uso del espacio, conocidas como relaciones espaciales. Los objetivos intermedios están relacionados con el diseño y desarrollo de las experiencias `en la naturaleza': cambios mínimos como sea posible en el escenario, definición de tareas sociales, desarrollo gradual de una plataforma robótica lo más transparente posible desde el lado robótico. Inicialmente, esta investigación presenta varios estudios preliminares de interacción humano-robot (HRI) con el robot REEM de PAL Robotics en el Museo de Ciencias CosmoCaixa de Barcelona. Basado en ciertas teorías sobre el uso del espacio como una forma de comunicación o interacción social, la tarea en este estudio con el robot comercial es como guía de museo, tanto cuando el grupo estaba en movimiento (es decir, cuando estaba siendo guiado) como cuando el grupo estaba en un lugar estático. Además, se llevó a cabo un segundo estudio de HRI con un robot REEM que realizaba la tarea de un asistente de profesor para analizar la percepción de la presencia e identidad social del robot. Asimismo, se presenta el desarrollo de una plataforma robótica, conocida como MASHI, para el estudio de la HRI. En función del servicio que debe completar el robot, las mejoras en la plataforma robótica experimental (estructura, morfología, cabeza, cara, brazos) se llevaron a cabo en ciclos continuos siguiendo el desarrollo de las experiencias de HRI. Esta estructura debe mantenerse lo más simple posible para que sea 'transparente' en el estudio de HRI social. A continuación, se presenta el estudio de campo de la interacción social humano-robot con el robot MASHI con el papel de guía de exposición en un centro cultural. Con base en técnicas de observación directa, se realiza un estudio de las diferentes relaciones espaciales que se generan cuando un robot interactúa con una persona o grupos de personas. Finalmente, se introduce un enfoque novedoso para representar las relaciones espaciales de la HRI de forma cualitativa para las experiencias futuras. En este estudio final, analizamos diferentes arreglos espaciales generados en un escenario social con un robot con el rol de guía. Como conclusión principal, se puede afirmar que las personas siguen normas sociales, en forma de relaciones espaciales, cuando interactúan con un robot que brinda un servicio social en un espacio público. Los niños, sin embargo, desafían recurrentemente estas normas sociales, probablemente porque están aprendiendo constantemente sobre las normas que regulan nuestra convivencia. Las relaciones espaciales se refuerzan claramente cuando el rol asignado al robot es más explícito y entendido por las personas. Las relaciones espaciales pueden verse afectadas por las características del entorno, ya sea por el espacio disponible o por los elementos dispuestos en él, así como por el número de personas que lo habitan. En general, esta disertación señala que el servicio prestado, y su comprensión del lado del usuario, es más importante que las habilidades robóticas de la plataforma robótica con el fin de mejorar las experiencias del usuario en entornos públicosPostprint (published version

Human-robot Interaction For Multi-robot Systems

Designing an effective human-robot interaction paradigm is particularly important for complex tasks such as multi-robot manipulation that require the human and robot to work together in a tightly coupled fashion. Although increasing the number of robots can expand the area that the robots can cover within a bounded period of time, a poor human-robot interface will ultimately compromise the performance of the team of robots. However, introducing a human operator to the team of robots, does not automatically improve performance due to the difficulty of teleoperating mobile robots with manipulators. The human operator’s concentration is divided not only among multiple robots but also between controlling each robot’s base and arm. This complexity substantially increases the potential neglect time, since the operator’s inability to effectively attend to each robot during a critical phase of the task leads to a significant degradation in task performance. There are several proven paradigms for increasing the efficacy of human-robot interaction: 1) multimodal interfaces in which the user controls the robots using voice and gesture; 2) configurable interfaces which allow the user to create new commands by demonstrating them; 3) adaptive interfaces which reduce the operator’s workload as necessary through increasing robot autonomy. This dissertation presents an evaluation of the relative benefits of different types of user interfaces for multi-robot systems composed of robots with wheeled bases and three degree of freedom arms. It describes a design for constructing low-cost multi-robot manipulation systems from off the shelf parts. User expertise was measured along three axes (navigation, manipulation, and coordination), and participants who performed above threshold on two out of three dimensions on a calibration task were rated as expert. Our experiments reveal that the relative expertise of the user was the key determinant of the best performing interface paradigm for that user, indicating that good user modiii eling is essential for designing a human-robot interaction system that will be used for an extended period of time. The contributions of the dissertation include: 1) a model for detecting operator distraction from robot motion trajectories; 2) adjustable autonomy paradigms for reducing operator workload; 3) a method for creating coordinated multi-robot behaviors from demonstrations with a single robot; 4) a user modeling approach for identifying expert-novice differences from short teleoperation traces

Exploring Robot Teleoperation in Virtual Reality

This thesis presents research on VR-based robot teleoperation with a focus on remote environment visualisation in virtual reality, the effects of remote environment reconstruction scale in virtual reality on the human-operator's ability to control the robot and human-operator's visual attention patterns when teleoperating a robot from virtual reality. A VR-based robot teleoperation framework was developed, it is compatible with various robotic systems and cameras, allowing for teleoperation and supervised control with any ROS-compatible robot and visualisation of the environment through any ROS-compatible RGB and RGBD cameras. The framework includes mapping, segmentation, tactile exploration, and non-physically demanding VR interface navigation and controls through any Unity-compatible VR headset and controllers or haptic devices. Point clouds are a common way to visualise remote environments in 3D, but they often have distortions and occlusions, making it difficult to accurately represent objects' textures. This can lead to poor decision-making during teleoperation if objects are inaccurately represented in the VR reconstruction. A study using an end-effector-mounted RGBD camera with OctoMap mapping of the remote environment was conducted to explore the remote environment with fewer point cloud distortions and occlusions while using a relatively small bandwidth. Additionally, a tactile exploration study proposed a novel method for visually presenting information about objects' materials in the VR interface, to improve the operator's decision-making and address the challenges of point cloud visualisation. Two studies have been conducted to understand the effect of virtual world dynamic scaling on teleoperation flow. The first study investigated the use of rate mode control with constant and variable mapping of the operator's joystick position to the speed (rate) of the robot's end-effector, depending on the virtual world scale. The results showed that variable mapping allowed participants to teleoperate the robot more effectively but at the cost of increased perceived workload. The second study compared how operators used a virtual world scale in supervised control, comparing the virtual world scale of participants at the beginning and end of a 3-day experiment. The results showed that as operators got better at the task they as a group used a different virtual world scale, and participants' prior video gaming experience also affected the virtual world scale chosen by operators. Similarly, the human-operator's visual attention study has investigated how their visual attention changes as they become better at teleoperating a robot using the framework. The results revealed the most important objects in the VR reconstructed remote environment as indicated by operators' visual attention patterns as well as their visual priorities shifts as they got better at teleoperating the robot. The study also demonstrated that operators’ prior video gaming experience affects their ability to teleoperate the robot and their visual attention behaviours

Mobile robot teleoperation through eye-gaze (telegaze)

In most teleoperation applications the human operator is required to monitor the status of the robot, as well as, issue controlling commands for the whole duration of the operation. Using a vision based feedback system, monitoring the robot requires the operator to look at a continuous stream of images displayed on an interaction screen. The eyes of the operator therefore, are fully engaged in monitoring and the hands in controlling. Since the eyes of the operator are engaged in monitoring anyway, inputs from their gaze can be used to aid in controlling. This frees the hands of the operator, either partially or fully, from controlling which can then be used to perform any other necessary tasks. However, the challenge here lies in distinguishing between the inputs that can be used for controlling and the inputs that can be used for monitoring. In mobile robot teleoperation, controlling is mainly composed of issuing locomotion commands to drive the robot. Monitoring on the other hand, is looking where the robot goes and looking for any obstacles in the route. Interestingly, there exist a strong correlation between human's gazing behaviours and their moving intentions. This correlation has been exploited in this thesis to investigate novel means for mobile robot teleoperation through eye-gaze, which has been named TeleGaze for short

User modelling for robotic companions using stochastic context-free grammars

Creating models about others is a sophisticated human ability that robotic companions need to develop in order to have successful interactions. This thesis proposes user modelling frameworks to personalise the interaction between a robot and its user and devises novel scenarios where robotic companions may apply these user modelling techniques. We tackle the creation of user models in a hierarchical manner, using a streamlined version of the Hierarchical Attentive Multiple-Models for Execution and Recognition (HAMMER) architecture to detect low-level user actions and taking advantage of Stochastic Context-Free Grammars (SCFGs) to instantiate higher-level models which recognise uncertain and recursive sequences of low-level actions. We discuss a couple of distinct scenarios for robotic companions: a humanoid sidekick for power-wheelchair users and a companion of hospital patients. Next, we address the limitations of the previous scenarios by applying our user modelling techniques and designing two further scenarios that fully take advantage of the user model. These scenarios are: a wheelchair driving tutor which models the user abilities, and the musical collaborator which learns the preferences of its users. The methodology produced interesting results in all scenarios: users preferred the actual robot over a simulator as a wheelchair sidekick. Hospital patients rated positively their interactions with the companion independently of their age. Moreover, most users agreed that the music collaborator had become a better accompanist with our framework. Finally, we observed that users' driving performance improved when the robotic tutor instructed them to repeat a task. As our workforce ages and the care requirements in our society grow, robots will need to play a role in helping us lead better lives. This thesis shows that, through the use of SCFGs, adaptive user models may be generated which then can be used by robots to assist their users.Open Acces