Robots that Anticipate Pain: Anticipating Physical Perturbations from Visual Cues through Deep Predictive Models

abstract: To ensure system integrity, robots need to proactively avoid any unwanted physical perturbation that may cause damage to the underlying hardware. In this thesis work, we investigate a machine learning approach that allows robots to anticipate impending physical perturbations from perceptual cues. In contrast to other approaches that require knowledge about sources of perturbation to be encoded before deployment, our method is based on experiential learning. Robots learn to associate visual cues with subsequent physical perturbations and contacts. In turn, these extracted visual cues are then used to predict potential future perturbations acting on the robot. To this end, we introduce a novel deep network architecture which combines multiple sub- networks for dealing with robot dynamics and perceptual input from the environment. We present a self-supervised approach for training the system that does not require any labeling of training data. Extensive experiments in a human-robot interaction task show that a robot can learn to predict physical contact by a human interaction partner without any prior information or labeling. Furthermore, the network is able to successfully predict physical contact from either depth stream input or traditional video input or using both modalities as input.Dissertation/ThesisMasters Thesis Computer Science 201

Spatio-Temporal Avoidance of Predicted Occupancy in Human-Robot Collaboration

This paper addresses human-robot collaboration (HRC) challenges of integrating predictions of human activity to provide a proactive-n-reactive response capability for the robot. Prior works that consider current or predicted human poses as static obstacles are too nearsighted or too conservative in planning, potentially causing delayed robot paths. Alternatively, time-varying prediction of human poses would enable robot paths that avoid anticipated human poses, synchronized dynamically in time and space. Herein, a proactive path planning method, denoted STAP, is presented that uses spatiotemporal human occupancy maps to find robot trajectories that anticipate human movements, allowing robot passage without stopping. In addition, STAP anticipates delays from robot speed restrictions required by ISO/TS 15066 speed and separation monitoring (SSM). STAP also proposes a sampling-based planning algorithm based on RRT* to solve the spatio-temporal motion planning problem and find paths of minimum expected duration. Experimental results show STAP generates paths of shorter duration and greater average robot-human separation distance throughout tasks. Additionally, STAP more accurately estimates robot trajectory durations in HRC, which are useful in arriving at proactive-n-reactive robot sequencing.Comment: 7 pages, 7 figures. Accepted at IEEE ROMAN 202

Behavioural strategy for indoor mobile robot navigation in dynamic environments

PhD ThesisDevelopment of behavioural strategies for indoor mobile navigation has become a challenging and practical issue in a cluttered indoor environment, such as a hospital or factory, where there are many static and moving objects, including humans and other robots, all of which trying to complete their own specific tasks; some objects may be moving in a similar direction to the robot, whereas others may be moving in the opposite direction. The key requirement for any mobile robot is to avoid colliding with any object which may prevent it from reaching its goal, or as a consequence bring harm to any individual within its workspace. This challenge is further complicated by unobserved objects suddenly appearing in the robots path, particularly when the robot crosses a corridor or an open doorway. Therefore the mobile robot must be able to anticipate such scenarios and manoeuvre quickly to avoid collisions. In this project, a hybrid control architecture has been designed to navigate within dynamic environments. The control system includes three levels namely: deliberative, intermediate and reactive, which work together to achieve short, fast and safe navigation. The deliberative level creates a short and safe path from the current position of the mobile robot to its goal using the wavefront algorithm, estimates the current location of the mobile robot, and extracts the region from which unobserved objects may appear. The intermediate level links the deliberative level and the reactive level, that includes several behaviours for implementing the global path in such a way to avoid any collision. In avoiding dynamic obstacles, the controller has to identify and extract obstacles from the sensor data, estimate their speeds, and then regular its speed and direction to minimize the collision risk and maximize the speed to the goal. The velocity obstacle approach (VO) is considered an easy and simple method for avoiding dynamic obstacles, whilst the collision cone principle is used to detect the collision situation between two circular-shaped objects. However the VO approach has two challenges when applied in indoor environments. The first challenge is extraction of collision cones of non-circular objects from sensor data, in which applying fitting circle methods generally produces large and inaccurate collision cones especially for line-shaped obstacle such as walls. The second challenge is that the mobile robot cannot sometimes move to its goal because all its velocities to the goal are located within collision cones. In this project, a method has been demonstrated to extract the colliii sion cones of circular and non-circular objects using a laser sensor, where the obstacle size and the collision time are considered to weigh the robot velocities. In addition the principle of the virtual obstacle was proposed to minimize the collision risk with unobserved moving obstacles. The simulation and experiments using the proposed control system on a Pioneer mobile robot showed that the mobile robot can successfully avoid static and dynamic obstacles. Furthermore the mobile robot was able to reach its target within an indoor environment without causing any collision or missing the target

Human Robot Motion: A Shared Effort Approach

International audienceThis paper is about Human Robot Motion (HRM), i.e. the study of how a robot should move among humans. This problem has often been solved by considering persons as moving obstacles, predicting their future trajectories and avoiding these trajectories. In contrast with such an approach, recent works have showed benefits of robots that can move and avoid collisions in a manner similar to persons, what we call human-like motion. One such benefit is that human-like motion was shown to reduce the planning effort for all persons in the environment, given that they tend to solve collision avoidance problems in similar ways. The effort required for avoiding a collision, however, is not shared equally between agents as it varies depending on factors such as visibility and crossing order. Thus, this work tackles HRM using the notion of motion effort and how it should be shared between the robot and the person in order to avoid collisions. To that end our approach learns a robot behavior using Reinforcement Learning that enables it to mutually solve the collision avoidance problem during our simulated trials

Multiple vehicle cooperation and collision avoidance in automated vehicles : Survey and an AI‑enabled conceptual framework

Prospective customers are becoming more concerned about safety and comfort as the automobile industry swings toward automated vehicles (AVs). A comprehensive evaluation of recent AVs collision data indicates that modern automated driving systems are prone to rear-end collisions, usually leading to multiple-vehicle collisions. Moreover, most investigations into severe traffic conditions are confined to single-vehicle collisions. This work reviewed diverse techniques of existing literature to provide planning procedures for multiple vehicle cooperation and collision avoidance (MVCCA) strategies in AVs while also considering their performance and social impact viewpoints. Firstly, we investigate and tabulate the existing MVCCA techniques associated with single-vehicle collision avoidance perspectives. Then, current achievements are extensively evaluated, challenges and flows are identified, and remedies are intelligently formed to exploit a taxonomy. This paper also aims to give readers an AI-enabled conceptual framework and a decision-making model with a concrete structure of the training network settings to bridge the gaps between current investigations. These findings are intended to shed insight into the benefits of the greater efficiency of AVs set-up for academics and policymakers. Lastly, the open research issues discussed in this survey will pave the way for the actual implementation of driverless automated traffic systems

Interaction-Aware Motion Planning for Automated Vehicles

Die Bewegungsplanung für automatisierte Fahrzeuge (AVs) in gemischtem Verkehr ist eine herausfordernde Aufgabe. Hierbei bezeichnet gemischter Verkehr, Verkehr bestehend aus von Menschen gefahrenen Fahrzeugen sowie automatisierten Fahrzeugen. Um die Komplexität der Aufgabe zu reduzieren, verwenden state-of-the-art Planungsansätze oft die vereinfachende Annahme, dass das zukünftige Verhalten umliegender Fahrzeuge unabhängig vom Plan des AVs vorhergesagt werden kann. Während die Trennung von Prädiktion und Planung für viele Verkehrssituationen eine hilfreiche Vereinfachung darstellt, werden hierbei Interaktionen zwischen den Verkehrsteilnehmern ignoriert, was besonders in interaktiven Verkehrssituationen zu suboptimalem, übermäßig konservativem Fahrverhalten führen kann. In dieser Arbeit werden zwei interaktionsbewusste Bewegungsplanungsalgorithmen vorgeschlagen, die in der Lage sind übermäßig konservatives Fahrverhalten zu reduzieren. Der Kernaspekt dieser Algorithmen ist, dass Prädiktion und Planung gleichzeitig gelöst werden. Mit diesen Algorithmen können anspruchsvolle Fahrmanöver, wie z. B. das Reißverschlussverfahren in dichtem Verkehr, durchgeführt werden, die mit state-of-the-art Planungsansätzen nicht möglich sind. Der erste Algorithmus basiert auf Methoden der Multi-Agenten-Planung. Interaktionen zwischen Verkehrsteilnehmern werden durch Optimierung gekoppelter Trajektorien mittels einer gemeinsamen Kostenfunktion approximiert. Das Kernstück des Algorithmus ist eine neuartige Multi-Agenten-Trajektorienplanungsformulierung, die auf gemischt-ganzzahliger quadratischer Programmierung (MIQP) basiert. Die Formulierung garantiert global optimale Lösungen und ist somit in der Lage das kombinatorische Problem zu lösen, welches kontinuierliche Methoden auf lokal optimale Lösungen beschränkt. Desweiteren kann durch den vorgestellten Ansatz ein manöverneutrales Verhalten erzeugt werden, das Manöverentscheidungen in ungewissen Situationen aufschieben kann. Der zweite Ansatz formuliert Interaktionen zwischen einem menschlichen Fahrer und einem AV als ein Stackelberg-Spiel. Im Gegensatz zu bestehenden Arbeiten kann der Algorithmus allgemeine nichtlineare Zustands- und Eingabebeschränkungen berücksichtigen. Desweiteren führen wir Mechanismen zur Integration von Kooperation und Rücksichtnahme in die Planung ein. Damit wird übermäßig aggressives Fahrverhalten verhindert, was in der Literatur als ein Problem interaktionsbewusster Planungsmethoden identifiziert wurde. Die Wirksamkeit, Robustheit und Echtzeitfähigkeit des Algorithmus wird durch numerische Experimente gezeigt