Detection, location and grasping objects using a stereo sensor on UAV in outdoor environments

The article presents a vision system for the autonomous grasping of objects with Unmanned Aerial Vehicles (UAVs) in real time. Giving UAVs the capability to manipulate objects vastly extends their applications, as they are capable of accessing places that are difficult to reach or even unreachable for human beings. This work is focused on the grasping of known objects based on feature models. The system runs in an on-board computer on a UAV equipped with a stereo camera and a robotic arm. The algorithm learns a feature-based model in an offline stage, then it is used online for detection of the targeted object and estimation of its position. This feature-based model was proved to be robust to both occlusions and the presence of outliers. The use of stereo cameras improves the learning stage, providing 3D information and helping to filter features in the online stage. An experimental system was derived using a rotary-wing UAV and a small manipulator for final proof of concept. The robotic arm is designed with three degrees of freedom and is lightweight due to payload limitations of the UAV. The system has been validated with different objects, both indoors and outdoor

Autonomous aerial robot for high-speed search and intercept applications

In recent years, high-speed navigation and environment interaction in the context of aerial robotics has become a field of interest for several academic and industrial research studies. In particular, Search and Intercept (SaI) applications for aerial robots pose a compelling research area due to their potential usability in several environments. Nevertheless, SaI tasks involve a challenging development regarding sensory weight, onboard computation resources, actuation design, and algorithms for perception and control, among others. In this work, a fully autonomous aerial robot for high-speed object grasping has been proposed. As an additional subtask, our system is able to autonomously pierce balloons located in poles close to the surface. Our first contribution is the design of the aerial robot at an actuation and sensory level consisting of a novel gripper design with additional sensors enabling the robot to grasp objects at high speeds. The second contribution is a complete software framework consisting of perception, state estimation, motion planning, motion control, and mission control in order to rapidly and robustly perform the autonomous grasping mission. Our approach has been validated in a challenging international competition and has shown outstanding results, being able to autonomously search, follow, and grasp a moving object at 6 m/s in an outdoor environment.Agencia Estatal de InvestigaciónKhalifa Universit

Learning to Fly by Crashing

How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see: https://youtu.be/u151hJaGKU

Pushbroom Stereo for High-Speed Navigation in Cluttered Environments

We present a novel stereo vision algorithm that is capable of obstacle detection on a mobile-CPU processor at 120 frames per second. Our system performs a subset of standard block-matching stereo processing, searching only for obstacles at a single depth. By using an onboard IMU and state-estimator, we can recover the position of obstacles at all other depths, building and updating a full depth-map at framerate. Here, we describe both the algorithm and our implementation on a high-speed, small UAV, flying at over 20 MPH (9 m/s) close to obstacles. The system requires no external sensing or computation and is, to the best of our knowledge, the first high-framerate stereo detection system running onboard a small UAV

RUR53: an Unmanned Ground Vehicle for Navigation, Recognition and Manipulation

This paper proposes RUR53: an Unmanned Ground Vehicle able to autonomously navigate through, identify, and reach areas of interest; and there recognize, localize, and manipulate work tools to perform complex manipulation tasks. The proposed contribution includes a modular software architecture where each module solves specific sub-tasks and that can be easily enlarged to satisfy new requirements. Included indoor and outdoor tests demonstrate the capability of the proposed system to autonomously detect a target object (a panel) and precisely dock in front of it while avoiding obstacles. They show it can autonomously recognize and manipulate target work tools (i.e., wrenches and valve stems) to accomplish complex tasks (i.e., use a wrench to rotate a valve stem). A specific case study is described where the proposed modular architecture lets easy switch to a semi-teleoperated mode. The paper exhaustively describes description of both the hardware and software setup of RUR53, its performance when tests at the 2017 Mohamed Bin Zayed International Robotics Challenge, and the lessons we learned when participating at this competition, where we ranked third in the Gran Challenge in collaboration with the Czech Technical University in Prague, the University of Pennsylvania, and the University of Lincoln (UK).Comment: This article has been accepted for publication in Advanced Robotics, published by Taylor & Franci

Design of an Anthropomorphic, Compliant, and Lightweight Dual Arm for Aerial Manipulation

This paper presents an anthropomorphic, compliant and lightweight dual arm manipulator designed and developed for aerial manipulation applications with multi-rotor platforms. Each arm provides four degrees of freedom in a human-like kinematic configuration for end effector positioning: shoulder pitch, roll and yaw, and elbow pitch. The dual arm, weighting 1.3 kg in total, employs smart servo actuators and a customized and carefully designed aluminum frame structure manufactured by laser cut. The proposed design reduces the manufacturing cost as no computer numerical control machined part is used. Mechanical joint compliance is provided in all the joints, introducing a compact spring-lever transmission mechanism between the servo shaft and the links, integrating a potentiometer for measuring the deflection of the joints. The servo actuators are partially or fully isolated against impacts and overloads thanks to the ange bearings attached to the frame structure that support the rotation of the links and the deflection of the joints. This simple mechanism increases the robustness of the arms and safety in the physical interactions between the aerial robot and the environment. The developed manipulator has been validated through different experiments in fixed base test-bench and in outdoor flight tests.Unión Europea H2020-ICT-2014- 644271Ministerio de Economía y Competitividad DPI2015-71524-RMinisterio de Economía y Competitividad DPI2017-89790-

Experimental Evaluation of a Team of Multiple Unmanned Aerial Vehicles for Cooperative Construction

Nº Artículo 9314142This article presents a team of multiple Unmanned Aerial Vehicles (UAVs) to perform cooperative missions for autonomous construction. In particular, the UAVs have to build a wall made of bricks that need to be picked and transported from different locations. First, we propose a novel architecture for multi-robot systems operating in outdoor and unstructured environments, where robustness and reliability play a key role. Then, we describe the design of our aerial platforms and grasping mechanisms to pick, transport and place bricks. The system was particularly developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC), where Challenge 2 consisted of building a wall cooperatively with multiple UAVs. However, our approach is more general and extensible to other multi-UAV applications involving physical interaction, like package delivery. We present not only our results in the final stage of MBZIRC, but also our simulations and field experiments throughout the previous months to the competition, where we tuned our system and assessed its performance