Constrained Collective Movement in Human-Robot Teams

This research focuses on improving human-robot co-navigation for teams of robots and humans navigating together as a unit while accomplishing a desired task. Frequently, the team’s co-navigation is strongly influenced by a predefined Standard Operating Procedure (SOP), which acts as a high-level guide for where agents should go and what they should do. In this work, I introduce the concept of Constrained Collective Movement (CCM) of a team to describe how members of the team perform inter-team and intra-team navigation to execute a joint task while balancing environmental and application-specific constraints. This work advances robots’ abilities to participate along side humans in applications such as urban search and rescue, firefighters searching for people in a burning building, and military teams performing a building clearing operation. Incorporating robots on such teams could reduce the number of human lives put in danger while increasing the team’s ability to conduct beneficial tasks such as carrying life saving equipment to stranded people. Most previous work on generating more complex collaborative navigation for human- robot teams focuses solely on using model-based methods. These methods usually suffer from the need for hard coding the rules to follow, which can require much time and domain knowledge and can lead to unnatural behavior. This dissertation investigates merging high-level model-based knowledge representation with low-level behavior cloning to achieve CCM of a human-robot team performing collaborative co-navigation. To evaluate the approach, experiments are performed in simulation with the detail-rich game design engine Unity. Experiments show that the designed approach can learn elements of high-level behaviors with accuracies up to 88%. Additionally, the approach is shown to learn low-level robot control behaviors with accuracies up to 89%. To the best of my knowledge, this is the first attempt to blend classical AI methods with state-of-the-art machine learning methods for human-robot team collaborative co-navigation. This not only allows for better human-robot team co-navigation, but also has implications for improving other teamwork based human-robot applications such as joint manufacturing and social assistive robotics

Data efficiency in imitation learning with a focus on object manipulation

Imitation is a natural human behaviour that helps us learn new skills. Modelling this behaviour in robots, however, has many challenges. This thesis investigates the challenge of handling the expert demonstrations in an efficient way, so as to minimise the number of demonstrations required for robots to learn. To achieve this, it focuses on demonstration data efficiency at various steps of the imitation process. Specifically, it presents new methodologies that offer ways to acquire, augment and combine demonstrations in order to improve the overall imitation process. Firstly, the thesis explores an inexpensive and non-intrusive way of acquiring dexterous human demonstrations. Human hand actions are quite complex, especially when they involve object manipulation. The proposed framework tackles this by using a camera to capture the hand information and then retargeting it to a dexterous hand model. It does this by combining inverse kinematics with stochastic optimisation. The demonstrations collected with this framework can then be used in the imitation process. Secondly, the thesis presents a novel way to apply data augmentation to demonstrations. The main difficulty of augmenting demonstrations is that their trajectorial nature can make them unsuccessful. Whilst previous works require additional knowledge about the task or demonstrations to achieve this, this method performs augmentation automatically. To do this, it introduces a correction network that corrects the augmentations based on the distribution of the original experts. Lastly, the thesis investigates data efficiency in a multi-task scenario where it additionally proposes a data combination method. Its aim is to automatically divide a set of tasks into sub-behaviours. Contrary to previous works, it does this without any additional knowledge about the tasks. To achieve this, it uses both task-specific and shareable modules. This minimises negative transfer and allows for the method to be applied to various task sets with different commonalities.Open Acces

Mapping and Semantic Perception for Service Robotics

Para realizar una tarea, los robots deben ser capaces de ubicarse en el entorno. Si un robot no sabe dónde se encuentra, es imposible que sea capaz de desplazarse para alcanzar el objetivo de su tarea. La localización y construcción de mapas simultánea, llamado SLAM, es un problema estudiado en la literatura que ofrece una solución a este problema. El objetivo de esta tesis es desarrollar técnicas que permitan a un robot comprender el entorno mediante la incorporación de información semántica. Esta información también proporcionará una mejora en la localización y navegación de las plataformas robóticas. Además, también demostramos cómo un robot con capacidades limitadas puede construir de forma fiable y eficiente los mapas semánticos necesarios para realizar sus tareas cotidianas.El sistema de construcción de mapas presentado tiene las siguientes características: En el lado de la construcción de mapas proponemos la externalización de cálculos costosos a un servidor en nube. Además, proponemos métodos para registrar información semántica relevante con respecto a los mapas geométricos estimados. En cuanto a la reutilización de los mapas construidos, proponemos un método que combina la construcción de mapas con la navegación de un robot para explorar mejor un entorno y disponer de un mapa semántico con los objetos relevantes para una misión determinada.En primer lugar, desarrollamos un algoritmo semántico de SLAM visual que se fusiona los puntos estimados en el mapa, carentes de sentido, con objetos conocidos. Utilizamos un sistema monocular de SLAM basado en un EKF (Filtro Extendido de Kalman) centrado principalmente en la construcción de mapas geométricos compuestos únicamente por puntos o bordes; pero sin ningún significado o contenido semántico asociado. El mapa no anotado se construye utilizando sólo la información extraída de una secuencia de imágenes monoculares. La parte semántica o anotada del mapa -los objetos- se estiman utilizando la información de la secuencia de imágenes y los modelos de objetos precalculados. Como segundo paso, mejoramos el método de SLAM presentado anteriormente mediante el diseño y la implementación de un método distribuido. La optimización de mapas y el almacenamiento se realiza como un servicio en la nube, mientras que el cliente con poca necesidad de computo, se ejecuta en un equipo local ubicado en el robot y realiza el cálculo de la trayectoria de la cámara. Los ordenadores con los que está equipado el robot se liberan de la mayor parte de los cálculos y el único requisito adicional es una conexión a Internet.El siguiente paso es explotar la información semántica que somos capaces de generar para ver cómo mejorar la navegación de un robot. La contribución en esta tesis se centra en la detección 3D y en el diseño e implementación de un sistema de construcción de mapas semántico.A continuación, diseñamos e implementamos un sistema de SLAM visual capaz de funcionar con robustez en entornos poblados debido a que los robots de servicio trabajan en espacios compartidos con personas. El sistema presentado es capaz de enmascarar las zonas de imagen ocupadas por las personas, lo que aumenta la robustez, la reubicación, la precisión y la reutilización del mapa geométrico. Además, calcula la trayectoria completa de cada persona detectada con respecto al mapa global de la escena, independientemente de la ubicación de la cámara cuando la persona fue detectada.Por último, centramos nuestra investigación en aplicaciones de rescate y seguridad. Desplegamos un equipo de robots en entornos que plantean múltiples retos que implican la planificación de tareas, la planificación del movimiento, la localización y construcción de mapas, la navegación segura, la coordinación y las comunicaciones entre todos los robots. La arquitectura propuesta integra todas las funcionalidades mencionadas, asi como varios aspectos de investigación novedosos para lograr una exploración real, como son: localización basada en características semánticas-topológicas, planificación de despliegue en términos de las características semánticas aprendidas y reconocidas, y construcción de mapas.In order to perform a task, robots need to be able to locate themselves in the environment. If a robot does not know where it is, it is impossible for it to move, reach its goal and complete the task. Simultaneous Localization and Mapping, known as SLAM, is a problem extensively studied in the literature for enabling robots to locate themselves in unknown environments. The goal of this thesis is to develop and describe techniques to allow a service robot to understand the environment by incorporating semantic information. This information will also provide an improvement in the localization and navigation of robotic platforms. In addition, we also demonstrate how a simple robot can reliably and efficiently build the semantic maps needed to perform its quotidian tasks. The mapping system as built has the following features. On the map building side we propose the externalization of expensive computations to a cloud server. Additionally, we propose methods to register relevant semantic information with respect to the estimated geometrical maps. Regarding the reuse of the maps built, we propose a method that combines map building with robot navigation to better explore a room in order to obtain a semantic map with the relevant objects for a given mission. Firstly, we develop a semantic Visual SLAM algorithm that merges traditional with known objects in the estimated map. We use a monocular EKF (Extended Kalman Filter) SLAM system that has mainly been focused on producing geometric maps composed simply of points or edges but without any associated meaning or semantic content. The non-annotated map is built using only the information extracted from an image sequence. The semantic or annotated parts of the map –the objects– are estimated using the information in the image sequence and the precomputed object models. As a second step we improve the EKF SLAM presented previously by designing and implementing a visual SLAM system based on a distributed framework. The expensive map optimization and storage is allocated as a service in the Cloud, while a light camera tracking client runs on a local computer. The robot’s onboard computers are freed from most of the computation, the only extra requirement being an internet connection. The next step is to exploit the semantic information that we are able to generate to see how to improve the navigation of a robot. The contribution of this thesis is focused on 3D sensing which we use to design and implement a semantic mapping system. We then design and implement a visual SLAM system able to perform robustly in populated environments due to service robots work in environments where people are present. The system is able to mask the image regions occupied by people out of the rigid SLAM pipeline, which boosts the robustness, the relocation, the accuracy and the reusability of the geometrical map. In addition, it estimates the full trajectory of each detected person with respect to the scene global map, irrespective of the location of the moving camera at the point when the people were imaged. Finally, we focus our research on rescue and security applications. The deployment of a multirobot team in confined environments poses multiple challenges that involve task planning, motion planning, localization and mapping, safe navigation, coordination and communications among all the robots. The architecture integrates, jointly with all the above-mentioned functionalities, several novel features to achieve real exploration: localization based on semantic-topological features, deployment planning in terms of the semantic features learned and recognized, and map building.<br /

Experience based action planning for environmental manipulation in autonomous robotic systems

The ability for autonomous robots to plan action sequences in order to manipulate their environment to achieve a specific goal is of vital importance for agents which are deployed in a vast number of situations. From domestic care robots to autonomous swarms of search and rescue robots there is a need for agents to be able to study, reason about, and manipulate their environment without the oversight of human operators. As these robots are typically deployed in areas inhabited and organised by humans it is likely that they will encounter similar objects when going about their duties, and in many cases the objects encountered are likely to be arranged in similar ways relative to one another. Manipulation of the environment is an incredibly complex task requiring vast amounts of computation to generate a suitable state of actions for even the simplest of tasks. To this end we explore the application of memory based systems to environment manipulation planning. We propose new search techniques targeted at the problem of environmental manipulation for search and rescue, and recall techniques aimed at allowing more complex planning to take place with lower computational cost. We explore these ideas from the perspective of autonomous robotic systems deployed for search and rescue, however the techniques presented would be equally valid for robots in other areas, or for virtual agents interacting with cyber-physical systems

Utilizing Reinforcement Learning and Computer Vision in a Pick-And-Place Operation for Sorting Objects in Motion

This master's thesis studies the implementation of advanced machine learning (ML) techniques in industrial automation systems, focusing on applying machine learning to enable and evolve autonomous sorting capabilities in robotic manipulators. In particular, Inverse Kinematics (IK) and Reinforcement Learning (RL) are investigated as methods for controlling a UR10e robotic arm for pick-and-place of moving objects on a conveyor belt within a small-scale sorting facility. A camera-based computer vision system applying YOLOv8 is used for real-time object detection and instance segmentation. Perception data is utilized to ascertain optimal grip points, specifically through an implemented algorithm that outputs optimal grip position, angle, and width. As the implemented system includes testing and evaluation on a physical system, the intricacies of hardware control, specifically the reverse engineering of an OnRobot RG6 gripper is elaborated as part of this study. The system is implemented on the Robotic Operating System (ROS), and its design is in particular driven by high modularity and scalability in mind. The camera-based vision system serves as the primary input, while the robot control is the output. The implemented system design allows for the evaluation of motion control employing both IK and RL. Computation of IK is conducted via MoveIt2, while the RL model is trained and computed in NVIDIA Isaac Sim. The high-level control of the robotic manipulator was accomplished with use of Proximal Policy Optimization (PPO). The main result of the research is a novel reward function for the pick-and-place operation that takes into account distance and orientation from the target object. In addition, the provided system administers task control by independently initializing pick-and-place operation phases for each environment. The findings demonstrate that PPO was able to significantly enhance the velocity, accuracy, and adaptability of industrial automation. Our research shows that accurate control of the robot arm can be reached by training the PPO Model purely by applying a digital twin simulation

The Future of Humanoid Robots

This book provides state of the art scientific and engineering research findings and developments in the field of humanoid robotics and its applications. It is expected that humanoids will change the way we interact with machines, and will have the ability to blend perfectly into an environment already designed for humans. The book contains chapters that aim to discover the future abilities of humanoid robots by presenting a variety of integrated research in various scientific and engineering fields, such as locomotion, perception, adaptive behavior, human-robot interaction, neuroscience and machine learning. The book is designed to be accessible and practical, with an emphasis on useful information to those working in the fields of robotics, cognitive science, artificial intelligence, computational methods and other fields of science directly or indirectly related to the development and usage of future humanoid robots. The editor of the book has extensive R&D experience, patents, and publications in the area of humanoid robotics, and his experience is reflected in editing the content of the book