Safety-Aware Human-Robot Collaborative Transportation and Manipulation with Multiple MAVs

Human-robot interaction will play an essential role in various industries and daily tasks, enabling robots to effectively collaborate with humans and reduce their physical workload. Most of the existing approaches for physical human-robot interaction focus on collaboration between a human and a single ground robot. In recent years, very little progress has been made in this research area when considering aerial robots, which offer increased versatility and mobility compared to their grounded counterparts. This paper proposes a novel approach for safe human-robot collaborative transportation and manipulation of a cable-suspended payload with multiple aerial robots. We leverage the proposed method to enable smooth and intuitive interaction between the transported objects and a human worker while considering safety constraints during operations by exploiting the redundancy of the internal transportation system. The key elements of our system are (a) a distributed payload external wrench estimator that does not rely on any force sensor; (b) a 6D admittance controller for human-aerial-robot collaborative transportation and manipulation; (c) a safety-aware controller that exploits the internal system redundancy to guarantee the execution of additional tasks devoted to preserving the human or robot safety without affecting the payload trajectory tracking or quality of interaction. We validate the approach through extensive simulation and real-world experiments. These include as well the robot team assisting the human in transporting and manipulating a load or the human helping the robot team navigate the environment. To the best of our knowledge, this work is the first to create an interactive and safety-aware approach for quadrotor teams that physically collaborate with a human operator during transportation and manipulation tasks.Comment: Guanrui Li and Xinyang Liu contributed equally to this pape

Application of Simultaneous Localization and Mapping Algorithms for Haptic Teleoperation of Aerial Vehicles

In this thesis, a new type of haptic teleoperator system for remote control of Unmanned Aerial Vehicles (UAVs) has been developed, where the Simultaneous Localization and Mapping (SLAM) algorithms are implemented for the purpose of generating the haptic feedback. Specifically, the haptic feedback is provided to the human operator through interaction with artificial potential field built around the obstacles in the virtual environment which is located at the master site of the teleoperator system. The obstacles in the virtual environment replicate essential features of the actual remote environment where the UAV executes its tasks. The state of the virtual environment is generated and updated in real time using Extended Kalman Filter SLAM algorithms based on measurements performed by the UAV in the actual remote environment. Two methods for building haptic feedback from SLAM algorithms have been developed. The basic SLAM-based haptic feedback algorithm uses fixed size potential field around the obstacles, while the robust SLAM-based haptic feedback algorithm changes the size of potential field around the obstacle depending on the amount of uncertainty in obstacle location, which is represented by the covariance estimate provided by EKF. Simulations and experimental results are presented that evaluate the performance of the proposed teleoperator system

Middleware and Architecture for Advanced Applications of Cyber-physical Systems

In this thesis, we address issues related to middleware, architecture and applications of cyber-physical systems. The first problem we address is the cross-layer design of cyber-physical systems to cope with interactions between the cyber layer and the physical layer in a dynamic environment. We propose a bi-directional middleware that allows the optimal utilization of the common resources for the benefit of either or both the layers in order to obtain overall system performance. The case study of network connectivity preservation in a vehicular formation illustrates how this approach can be applied to a particular situation where the network connectivity drives the application layer. Next we address another aspect of cross-layer impact: the problem that arises when network performance, in this case delay performance, affects control system performance. We propose a two-pronged approach involving a flexible adaptive model identification algorithm with outlier rejection, which in turn uses an adaptive system model to detect and reject outliers, thus shielding the estimation algorithm and thereby improving reliability. We experimentally demonstrate that the outlier rejection approach which intercepts and filters the data, combined with simultaneous model adaptation, can result in improved performance of Model Predictive Control in the vehicular testbed. Then we turn to two advanced applications of cyber-physical systems. First, we address the problem of security of cyber-physical systems. We consider the context of an intelligent transportation system in which a malicious sensor node manipulates the position data of one of the autonomous cars to deviate from a safe trajectory and collide with other cars. In order to secure the safety of such systems where sensor measurements are compromised, we employ the procedure of “dynamic watermarking”. This procedure enables an honest node in the control loop to detect the existence of a malicious node within the feedback loop. We demonstrate in the testbed that dynamic watermarking can indeed protect cars against collisions even in the presence of sensor attacks. The second application of cyber-physical systems that we consider is cyber-manufacturing which is an origami-type laser-based custom manufacturing machine employing folding and cutting of sheet material to manufacture 3D objects. We have developed such a system for use in a laser-based autonomous custom manufacturing machine equipped with real-time sensing and control. The basic elements in the architecture are a laser processing machine, a sensing system to estimate the state of the workpiece, a control system determining control inputs for a laser system based on the estimated data, a robotic arm manipulating the workpiece in the work space, and middleware supporting the communication among the systems. We demonstrate automated 3D laser cutting and bending to fabricate a 3D product as an experimental result. Lastly, we address the problem of traffic management of an unmanned aerial system. In an effort to improve the performance of the traffic management for unmanned aircrafts, we propose a probability-based collision resolution algorithm. The proposed algorithm analyzes the planned trajectories to calculate their collision probabilities, and modifies individual drone starting times to reduce the probability of collision, while attempting to preserve high performance. Our simulation results demonstrate that the proposed algorithm improves the performance of the drone traffic management by guaranteeing high safety with low modification of the starting times

Hand-worn Haptic Interface for Drone Teleoperation

Drone teleoperation is usually accomplished using remote radio controllers, devices that can be hard to master for inexperienced users. Moreover, the limited amount of information fed back to the user about the robot's state, often limited to vision, can represent a bottleneck for operation in several conditions. In this work, we present a wearable interface for drone teleoperation and its evaluation through a user study. The two main features of the proposed system are a data glove to allow the user to control the drone trajectory by hand motion and a haptic system used to augment their awareness of the environment surrounding the robot. This interface can be employed for the operation of robotic systems in line of sight (LoS) by inexperienced operators and allows them to safely perform tasks common in inspection and search-and-rescue missions such as approaching walls and crossing narrow passages with limited visibility conditions. In addition to the design and implementation of the wearable interface, we performed a systematic study to assess the effectiveness of the system through three user studies (n = 36) to evaluate the users' learning path and their ability to perform tasks with limited visibility. We validated our ideas in both a simulated and a real-world environment. Our results demonstrate that the proposed system can improve teleoperation performance in different cases compared to standard remote controllers, making it a viable alternative to standard Human-Robot Interfaces.Comment: Accepted at the IEEE International Conference on Robotics and Automation (ICRA) 202

SwarmTouch: Tactile Interaction of Human with Impedance Controlled Swarm of Nano-Quadrotors

We propose a novel interaction strategy for a human-swarm communication when a human operator guides a formation of quadrotors with impedance control and receives vibrotactile feedback. The presented approach takes into account the human hand velocity and changes the formation shape and dynamics accordingly using impedance interlinks simulated between quadrotors, which helps to achieve a life-like swarm behavior. Experimental results with Crazyflie 2.0 quadrotor platform validate the proposed control algorithm. The tactile patterns representing dynamics of the swarm (extension or contraction) are proposed. The user feels the state of the swarm at his fingertips and receives valuable information to improve the controllability of the complex life-like formation. The user study revealed the patterns with high recognition rates. Subjects stated that tactile sensation improves the ability to guide the drone formation and makes the human-swarm communication much more interactive. The proposed technology can potentially have a strong impact on the human-swarm interaction, providing a new level of intuitiveness and immersion into the swarm navigation.Comment: \c{opyright} 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. arXiv admin note: substantial text overlap with arXiv:1909.0229

Master of Science

thesisThis thesis details the development of the Algorithmic Robotics Laboratory, its experimental software environment, and a case study featuring a novel hardware validation of optimal reciprocal collision avoidance. We constructed a robotics laboratory in both software and hardware in which to perform our experiments. This lab features a netted flying volume with motion capture and two custom quadrotors. Also, two experimental software architectures are developed for actuating both ground and aerial robots within a Linux Robot Operating System environment. The first of the frameworks is based upon a single finite state machine program which managed each aspect of the experiment. Concerns about the complexity and reconfigurability of the finite state machine prompted the development of a second framework. This final framework is a multimodal structure featuring programs which focus on these specific functions: State Estimation, Robot Drivers, Experimental Controllers, Inputs, Human Robot Interaction, and a program tailored to the specifics of the algorithm tested in the experiment. These modular frameworks were used to fulfill the mission of the Algorithmic Robotics Lab, in that they were developed to validate robotics algorithms in experiments that were previously only shown in simulation. A case study into collision avoidance was used to mark the foundation of the laboratory through the proving of an optimal reciprocal collision avoidance algorithm for the first time in hardware. In the case study, two human-controlled quadrotors were maliciously flown in colliding trajectories. Optimal reciprocal collision avoidance was demonstrated for the first time on completely independent agents with local sensing. The algorithm was shown to be robust to violations of its inherent assumptions about the dynamics of agents and the ability for those agents to sense imminent collisions. These experiments, in addition to the mathematical foundation of exponential convergence, submits th a t optimal reciprocal collision avoidance is a viable method for holonomic robots in both 2-D and 3-D with noisy sensing. A basis for the idea of reciprocal dance, a motion often seen in human collision avoidance, is also suggested in demonstration to be a product of uncertainty about the state of incoming agents. In the more than one hundred tests conducted in multiple environments, no midair collisions were ever produced