Learning Pose Estimation for UAV Autonomous Navigation and Landing Using Visual-Inertial Sensor Data

In this work, we propose a robust network-in-the-loop control system for autonomous navigation and landing of an Unmanned-Aerial-Vehicle (UAV). To estimate the UAV’s absolute pose, we develop a deep neural network (DNN) architecture for visual-inertial odometry, which provides a robust alternative to traditional methods. We first evaluate the accuracy of the estimation by comparing the prediction of our model to traditional visual-inertial approaches on the publicly available EuRoC MAV dataset. The results indicate a clear improvement in the accuracy of the pose estimation up to 25% over the baseline. Finally, we integrate the data-driven estimator in the closed-loop flight control system of Airsim, a simulator available as a plugin for Unreal Engine, and we provide simulation results for autonomous navigation and landing

Distributed Control for Collective Behaviour in Micro-unmanned Aerial Vehicles

Full version unavailable due to 3rd party copyright restrictions.The work presented herein focuses on the design of distributed autonomous controllers for collective behaviour of Micro-unmanned Aerial Vehicles (MAVs). Two alternative approaches to this topic are introduced: one based upon the Evolutionary Robotics (ER) paradigm, the other one upon flocking principles. Three computer simulators have been developed in order to carry out the required experiments, all of them having their focus on the modelling of fixed-wing aircraft flight dynamics. The employment of fixed-wing aircraft rather than the omni-directional robots typically employed in collective robotics significantly increases the complexity of the challenges that an autonomous controller has to face. This is mostly due to the strict motion constraints associated with fixed-wing platforms, that require a high degree of accuracy by the controller. Concerning the ER approach, the experimental setups elaborated have resulted in controllers that have been evolved in simulation with the following capabilities: (1) navigation across unknown environments, (2) obstacle avoidance, (3) tracking of a moving target, and (4) execution of cooperative and coordinated behaviours based on implicit communication strategies. The design methodology based upon flocking principles has involved tests on computer simulations and subsequent experimentation on real-world robotic platforms. A customised implementation of Reynolds’ flocking algorithm has been developed and successfully validated through flight tests performed with the swinglet MAV. It has been notably demonstrated how the Evolutionary Robotics approach could be successfully extended to the domain of fixed-wing aerial robotics, which has never received a great deal of attention in the past. The investigations performed have also shown that complex and real physics-based computer simulators are not a compulsory requirement when approaching the domain of aerial robotics, as long as proper autopilot systems (taking care of the ”reality gap” issue) are used on the real robots.EOARD (European Office of Aerospace Research & Development), euCognitio

Motion planning with dynamics awareness for long reach manipulation in aerial robotic systems with two arms

Human activities in maintenance of industrial plants pose elevated risks as well as significant costs due to the required shutdowns of the facility. An aerial robotic system with two arms for long reach manipulation in cluttered environments is presented to alleviate these constraints. The system consists of a multirotor with a long bar extension that incorporates a lightweight dual arm in the tip. This configuration allows aerial manipulation tasks even in hard-to-reach places. The objective of this work is the development of planning strategies to move the aerial robotic system with two arms for long reach manipulation in a safe and efficient way for both navigation and manipulation tasks. The motion planning problem is addressed considering jointly the aerial platform and the dual arm in order to achieve wider operating conditions. Since there exists a strong dynamical coupling between the multirotor and the dual arm, safety in obstacle avoidance will be assured by introducing dynamics awareness in the operation of the planner. On the other hand, the limited maneuverability of the system emphasizes the importance of energy and time efficiency in the generated trajectories. Accordingly, an adapted version of the optimal Rapidly-exploring Random Tree algorithm has been employed to guarantee their optimality. The resulting motion planning strategy has been evaluated through simulation in two realistic industrial scenarios, a riveting application and a chimney repairing task. To this end, the dynamics of the aerial robotic system with two arms for long reach manipulation has been properly modeled, and a distributed control scheme has been derived to complete the test bed. The satisfactory results of the simulations are presented as a first validation of the proposed approach.Unión Europea H2020-644271Ministerio de Ciencia, Innovación y Universidades DPI2014-59383-C2-1-

Gesture Based Control of Semi-Autonomous Vehicles

The objective of this investigation is to explore the use of hand gestures to control semi-autonomous vehicles, such as quadcopters, using realistic, physics based simulations. This involves identifying natural gestures to control basic functions of a vehicle, such as maneuvering and onboard equipment operation, and building simulations using the Unity game engine to investigate preferred use of those gestures. In addition to creating a realistic operating experience, human factors associated with limitations on physical hand motion and information management are also considered in the simulation development process. Testing with external participants using a recreational quadcopter simulation built in Unity was conducted to assess the suitability of the simulation and preferences between a joystick approach and the gesture-based approach. Initial feedback indicated that the simulation represented the actual vehicle performance well and that the joystick is preferred over the gesture-based approach. Improvements in the gesture-based control are documented as additional features in the simulation, such as basic maneuver training and additional vehicle positioning information, are added to assist the user to better learn the gesture-based interface and implementation of active control concepts to interpret and apply vehicle forces and torques. Tests were also conducted with an actual ground vehicle to investigate if knowledge and skill from the simulated environment transfers to a real-life scenario. To assess this, an immersive virtual reality (VR) simulation was built in Unity as a training environment to learn how to control a remote control car using gestures. This was then followed by a control of the actual ground vehicle. Observations and participant feedback indicated that range of hand movement and hand positions transferred well to the actual demonstration. This illustrated that the VR simulation environment provides a suitable learning experience, and an environment from which to assess human performance; thus, also validating the observations from earlier tests. Overall results indicate that the gesture-based approach holds promise given the emergence of new technology, but additional work needs to be pursued. This includes algorithms to process gesture data to provide more stable and precise vehicle commands and training environments to familiarize users with this new interface concept

Autonomous Drone Landings on an Unmanned Marine Vehicle using Deep Reinforcement Learning

This thesis describes with the integration of an Unmanned Surface Vehicle (USV) and an Unmanned Aerial Vehicle (UAV, also commonly known as drone) in a single Multi-Agent System (MAS). In marine robotics, the advantage offered by a MAS consists of exploiting the key features of a single robot to compensate for the shortcomings in the other. In this way, a USV can serve as the landing platform to alleviate the need for a UAV to be airborne for long periods time, whilst the latter can increase the overall environmental awareness thanks to the possibility to cover large portions of the prevailing environment with a camera (or more than one) mounted on it. There are numerous potential applications in which this system can be used, such as deployment in search and rescue missions, water and coastal monitoring, and reconnaissance and force protection, to name but a few. The theory developed is of a general nature. The landing manoeuvre has been accomplished mainly identifying, through artificial vision techniques, a fiducial marker placed on a flat surface serving as a landing platform. The raison d'etre for the thesis was to propose a new solution for autonomous landing that relies solely on onboard sensors and with minimum or no communications between the vehicles. To this end, initial work solved the problem while using only data from the cameras mounted on the in-flight drone. In the situation in which the tracking of the marker is interrupted, the current position of the USV is estimated and integrated into the control commands. The limitations of classic control theory used in this approached suggested the need for a new solution that empowered the flexibility of intelligent methods, such as fuzzy logic or artificial neural networks. The recent achievements obtained by deep reinforcement learning (DRL) techniques in end-to-end control in playing the Atari video-games suite represented a fascinating while challenging new way to see and address the landing problem. Therefore, novel architectures were designed for approximating the action-value function of a Q-learning algorithm and used to map raw input observation to high-level navigation actions. In this way, the UAV learnt how to land from high latitude without any human supervision, using only low-resolution grey-scale images and with a level of accuracy and robustness. Both the approaches have been implemented on a simulated test-bed based on Gazebo simulator and the model of the Parrot AR-Drone. The solution based on DRL was further verified experimentally using the Parrot Bebop 2 in a series of trials. The outcomes demonstrate that both these innovative methods are both feasible and practicable, not only in an outdoor marine scenario but also in indoor ones as well

Learning Pose Estimation for UAV Autonomous Navigation and Landing Using Visual-Inertial Sensor Data

In this work, we propose a robust network-in-the-loop control system for autonomous navigation and landing of an Unmanned-Aerial-Vehicle (UAV). To estimate the UAV’s absolute pose, we develop a deep neural network (DNN) architecture for visual-inertial odometry, which provides a robust alternative to traditional methods. We first evaluate the accuracy of the estimation by comparing the prediction of our model to traditional visual-inertial approaches on the publicly available EuRoC MAV dataset. The results indicate a clear improvement in the accuracy of the pose estimation up to 25% over the baseline. Finally, we integrate the data-driven estimator in the closed-loop flight control system of Airsim, a simulator available as a plugin for Unreal Engine, and we provide simulation results for autonomous navigation and landing

A Systemic Approach to Next Generation Infrastructure Data Elicitation and Planning Using Serious Gaming Methods

Infrastructure systems are vital to the functioning of our society and economy. However, these systems are increasingly complex and are more interdependent than ever, making them difficult to manage. In order to respond to increasing demand, environmental concerns, and natural and man-made threats, infrastructure systems have to adapt and transform. Traditional engineering design approaches and planning tools have proven to be inadequate when planning and managing these complex socio-technical system transitions. The design and implementation of next generation infrastructure systems require holistic methodologies, encompassing organizational and societal aspects in addition to technical factors. In order to do so, a serious gaming based risk assessment methodology is developed to assist infrastructure data elicitation and planning. The methodology combines the use of various models, commercial-off-the-shelf solutions and a gaming approach to aggregate the inputs of various subject matter experts (SMEs) to predict future system characteristics. The serious gaming based approach enables experts to obtain a thorough understanding of the complexity and interdependency of the system while offering a platform to experiment with various strategies and scenarios. In order to demonstrate its abilities, the methodology was applied to National Airspace System (NAS) overhaul and its transformation to Next Generation Air Transportation System (NextGen). The implemented methodology yielded a comprehensive safety assessment and data generation mechanism, embracing the social and technical aspects of the NAS transformation for the next 15 years

Internet of Unmanned Aerial Vehicles—A Multilayer Low-Altitude Airspace Model for Distributed UAV Traffic Management

The rapid adoption of Internet of Things (IoT) has encouraged the integration of new connected devices such as Unmanned Aerial Vehicles (UAVs) to the ubiquitous network. UAVs promise a pragmatic solution to the limitations of existing terrestrial IoT infrastructure as well as bring new means of delivering IoT services through a wide range of applications. Owning to their potential, UAVs are expected to soon dominate the low-altitude airspace over populated cities. This introduces new research challenges such as the safe management of UAVs operation under high traffic demands. This paper proposes a novel way of structuring the uncontrolled, low-altitude airspace, with the aim of addressing the complex problem of UAV traffic management at an abstract level. The work, hence, introduces a model of the airspace as a weighted multilayer network of nodes and airways and presents a set of experimental simulation results using three UAV traffic management heuristics