Learning-based wildfire tracking with unmanned aerial vehicles

This project attempts to design a path planning algorithm for a group of unmanned aerial vehicles (UAVs) to track multiple spreading wildfire zones on a wildland. Due to the physical limitations of UAVs, the wildland is partially observable. Thus, the fire spreading is difficult to model. An online training regression neural network using real-time UAV observation data is implemented for fire front positions prediction. The wildfire tracking with UAVs path planning algorithm is proposed by Q-learning. Various practical factors are considered by designing an appropriate cost function which can describe the tracking problem, such as importance of the moving targets, field of view of UAVs, spreading speed of fire zones, collision avoidance between UAVs, obstacle avoidance, and maximum information collection. To improve the computation efficiency, a vertices-based fire line feature extraction is used to reduce the fire line targets. Simulation results under various wind conditions validate the fire prediction accuracy and UAV tracking performance.Includes bibliographical references

Visual Detection of Small Unmanned Aircraft: Modeling the Limits of Human Pilots

The purpose of this study was to determine the key physical variables for visual detection of small, Unmanned Aircraft Systems (UAS), and to learn how these variables influence the ability of human pilots, in manned-aircraft operating between 60-knots to 160-knots in the airport terminal area, to see these small, unmanned aircraft in time to avoid a collision. The study also produced a set of probability curves for various operating scenarios, depicting the likelihood of visually detecting a small, unmanned aircraft in time to avoid colliding with it. The study used the known limits of human visual acuity, based on the mechanics of the human eye and previous research on human visual detection of distant objects, to define the human performance constraints for the visual search task. The results of the analysis suggest the probability of detection, in all cases modeled during the study, is far less than 50 percent. The probability of detection was well under 10 percent for small UAS aircraft similar to the products used by many recreational and hobby operators. The results of this study indicate the concept of see-and-avoid is not a reliable technique for collision prevention by manned-aircraft pilots when it comes to operating near small, unmanned aircraft. Since small, unmanned aircraft continue to appear in airspace where they do not belong, regulators and the industry need to accelerate the development and deployment of alternative methods for collision prevention between sUAS aircraft operations and manned-aircraft. The analysis effort for this study included the development of a new simulation model, building on existing models related to human visual detection of distant objects. This study extended existing research and used currently accepted standards to create a new model specifically tailored to small, unmanned aircraft detection. Since several input variables are not controllable, this study used a Monte Carlo simulation to provide a means for addressing the effects of uncertainty in the uncontrollable inputs that the previous models did not handle. The uncontrollable inputs include the airspeed and direction of flight for the unmanned aircraft, as well as the changing contrast between the unmanned aircraft target and its background as both the target aircraft and the observer encounter different background and lighting conditions. The reusable model created for this study will enable future research related to the visual detection of small, unmanned aircraft. It provides a new tool for studying the difficult task of visually detecting airborne, small, unmanned aircraft targets in time to maneuver clear of a possible collision with them. The study also tested alternative input values to the simulation model to explore how changes to small, unmanned aircraft features might improve the visual detectability of the unmanned aircraft by human pilots in manned aircraft. While these changes resulted in higher probabilities of detection, the overall detection probability remained very low thereby confirming the urgent need to build reliable collision avoidance capability into small UAS aircraft

Stochastic Real-time Optimal Control: A Pseudospectral Approach for Bearing-Only Trajectory Optimization

A method is presented to couple and solve the optimal control and the optimal estimation problems simultaneously, allowing systems with bearing-only sensors to maneuver to obtain observability for relative navigation without unnecessarily detracting from a primary mission. A fundamentally new approach to trajectory optimization and the dual control problem is developed, constraining polynomial approximations of the Fisher Information Matrix to provide an information gradient and allow prescription of the level of future estimation certainty required for mission accomplishment. Disturbances, modeling deficiencies, and corrupted measurements are addressed with recursive updating of the target estimate with an Unscented Kalman Filter and the optimal path with Radau pseudospectral collocation methods and sequential quadratic programming. The basic real-time optimal control (RTOC) structure is investigated, specifically addressing limitations of current techniques in this area that lose error integration. The resulting guidance method can be applied to any bearing-only system, such as submarines using passive sonar, anti-radiation missiles, or small UAVs seeking to land on power lines for energy harvesting. Methods and tools required for implementation are developed, including variable calculation timing and tip-tail blending for potential discontinuities. Validation is accomplished with simulation and flight test, autonomously landing a quadrotor helicopter on a wire

Unmanned Aerial Vehicles (UAVs): Collision Avoidance Systems and Approaches

Moving towards autonomy, unmanned vehicles rely heavily on state-of-the-art collision avoidance systems (CAS). A lot of work is being done to make the CAS as safe and reliable as possible, necessitating a comparative study of the recent work in this important area. The paper provides a comprehensive review of collision avoidance strategies used for unmanned vehicles, with the main emphasis on unmanned aerial vehicles (UAV). It is an in-depth survey of different collision avoidance techniques that are categorically explained along with a comparative analysis of the considered approaches w.r.t. different scenarios and technical aspects. This also includes a discussion on the use of different types of sensors for collision avoidance in the context of UAVs

Optimal Control and Coordination of Small UAVs for Vision-based Target Tracking

Small unmanned aerial vehicles (UAVs) are relatively inexpensive mobile sensing platforms capable of reliably and autonomously performing numerous tasks, including mapping, search and rescue, surveillance and tracking, and real-time monitoring. The general problem of interest that we address is that of using small, fixed-wing UAVs to perform vision-based target tracking, which entails that one or more camera-equipped UAVs is responsible for autonomously tracking a moving ground target. In the single-UAV setting, the underactuated UAV must maintain proximity and visibility of an unpredictable ground target while having a limited sensing region. We provide solutions from two different vantage points. The first regards the problem as a two-player zero-sum game and the second as a stochastic optimal control problem. The resulting control policies have been successfully field-tested, thereby verifying the efficacy of both approaches while highlighting the advantages of one approach over the other. When employing two UAVs, one can fuse vision-based measurements to improve the estimate of the target's position. Accordingly, the second part of this dissertation involves determining the optimal control policy for two UAVs to gather the best joint vision-based measurements of a moving ground target, which is first done in a simplified deterministic setting. The results in this setting show that the key optimal control strategy is the coordination of the UAVs' distances to the target and not of the viewing angles as is traditionally assumed, thereby showing the advantage of solving the optimal control problem over using heuristics. To generate a control policy robust to real-world conditions, we formulate the same control objective using higher order stochastic kinematic models. Since grid-based solutions are infeasible for a stochastic optimal control problem of this dimension, we employ a simulation-based dynamic programming technique that relies on regression to form the optimal policy maps, thereby demonstrating an effective solution to a multi-vehicle coordination problem that until recently seemed intractable on account of its dimension. The results show that distance coordination is again the key optimal control strategy and that the policy offers considerable advantages over uncoordinated optimal policies, namely reduced variability in the cost and a reduction in the severity and frequency of high-cost events

Autonomous Drone Landings on an Unmanned Marine Vehicle using Deep Reinforcement Learning

This thesis describes with the integration of an Unmanned Surface Vehicle (USV) and an Unmanned Aerial Vehicle (UAV, also commonly known as drone) in a single Multi-Agent System (MAS). In marine robotics, the advantage offered by a MAS consists of exploiting the key features of a single robot to compensate for the shortcomings in the other. In this way, a USV can serve as the landing platform to alleviate the need for a UAV to be airborne for long periods time, whilst the latter can increase the overall environmental awareness thanks to the possibility to cover large portions of the prevailing environment with a camera (or more than one) mounted on it. There are numerous potential applications in which this system can be used, such as deployment in search and rescue missions, water and coastal monitoring, and reconnaissance and force protection, to name but a few. The theory developed is of a general nature. The landing manoeuvre has been accomplished mainly identifying, through artificial vision techniques, a fiducial marker placed on a flat surface serving as a landing platform. The raison d'etre for the thesis was to propose a new solution for autonomous landing that relies solely on onboard sensors and with minimum or no communications between the vehicles. To this end, initial work solved the problem while using only data from the cameras mounted on the in-flight drone. In the situation in which the tracking of the marker is interrupted, the current position of the USV is estimated and integrated into the control commands. The limitations of classic control theory used in this approached suggested the need for a new solution that empowered the flexibility of intelligent methods, such as fuzzy logic or artificial neural networks. The recent achievements obtained by deep reinforcement learning (DRL) techniques in end-to-end control in playing the Atari video-games suite represented a fascinating while challenging new way to see and address the landing problem. Therefore, novel architectures were designed for approximating the action-value function of a Q-learning algorithm and used to map raw input observation to high-level navigation actions. In this way, the UAV learnt how to land from high latitude without any human supervision, using only low-resolution grey-scale images and with a level of accuracy and robustness. Both the approaches have been implemented on a simulated test-bed based on Gazebo simulator and the model of the Parrot AR-Drone. The solution based on DRL was further verified experimentally using the Parrot Bebop 2 in a series of trials. The outcomes demonstrate that both these innovative methods are both feasible and practicable, not only in an outdoor marine scenario but also in indoor ones as well

Swarm Robotics

Collectively working robot teams can solve a problem more efficiently than a single robot, while also providing robustness and flexibility to the group. Swarm robotics model is a key component of a cooperative algorithm that controls the behaviors and interactions of all individuals. The robots in the swarm should have some basic functions, such as sensing, communicating, and monitoring, and satisfy the following properties

Distributed Control for Collective Behaviour in Micro-unmanned Aerial Vehicles

Full version unavailable due to 3rd party copyright restrictions.The work presented herein focuses on the design of distributed autonomous controllers for collective behaviour of Micro-unmanned Aerial Vehicles (MAVs). Two alternative approaches to this topic are introduced: one based upon the Evolutionary Robotics (ER) paradigm, the other one upon flocking principles. Three computer simulators have been developed in order to carry out the required experiments, all of them having their focus on the modelling of fixed-wing aircraft flight dynamics. The employment of fixed-wing aircraft rather than the omni-directional robots typically employed in collective robotics significantly increases the complexity of the challenges that an autonomous controller has to face. This is mostly due to the strict motion constraints associated with fixed-wing platforms, that require a high degree of accuracy by the controller. Concerning the ER approach, the experimental setups elaborated have resulted in controllers that have been evolved in simulation with the following capabilities: (1) navigation across unknown environments, (2) obstacle avoidance, (3) tracking of a moving target, and (4) execution of cooperative and coordinated behaviours based on implicit communication strategies. The design methodology based upon flocking principles has involved tests on computer simulations and subsequent experimentation on real-world robotic platforms. A customised implementation of Reynolds’ flocking algorithm has been developed and successfully validated through flight tests performed with the swinglet MAV. It has been notably demonstrated how the Evolutionary Robotics approach could be successfully extended to the domain of fixed-wing aerial robotics, which has never received a great deal of attention in the past. The investigations performed have also shown that complex and real physics-based computer simulators are not a compulsory requirement when approaching the domain of aerial robotics, as long as proper autopilot systems (taking care of the ”reality gap” issue) are used on the real robots.EOARD (European Office of Aerospace Research & Development), euCognitio

Control and communication systems for automated vehicles cooperation and coordination

Mención Internacional en el título de doctorThe technological advances in the Intelligent Transportation Systems (ITS) are exponentially improving over the last century. The objective is to provide intelligent and innovative services for the different modes of transportation, towards a better, safer, coordinated and smarter transport networks. The Intelligent Transportation Systems (ITS) focus is divided into two main categories; the first is to improve existing components of the transport networks, while the second is to develop intelligent vehicles which facilitate the transportation process. Different research efforts have been exerted to tackle various aspects in the fields of the automated vehicles. Accordingly, this thesis is addressing the problem of multiple automated vehicles cooperation and coordination. At first, 3DCoAutoSim driving simulator was developed in Unity game engine and connected to Robot Operating System (ROS) framework and Simulation of Urban Mobility (SUMO). 3DCoAutoSim is an abbreviation for "3D Simulator for Cooperative Advanced Driver Assistance Systems (ADAS) and Automated Vehicles Simulator". 3DCoAutoSim was tested under different circumstances and conditions, afterward, it was validated through carrying-out several controlled experiments and compare the results against their counter reality experiments. The obtained results showed the efficiency of the simulator to handle different situations, emulating real world vehicles. Next is the development of the iCab platforms, which is an abbreviation for "Intelligent Campus Automobile". The platforms are two electric golf-carts that were modified mechanically, electronically and electrically towards the goal of automated driving. Each iCab was equipped with several on-board embedded computers, perception sensors and auxiliary devices, in order to execute the necessary actions for self-driving. Moreover, the platforms are capable of several Vehicle-to-Everything (V2X) communication schemes, applying three layers of control, utilizing cooperation architecture for platooning, executing localization systems, mapping systems, perception systems, and finally several planning systems. Hundreds of experiments were carried-out for the validation of each system in the iCab platform. Results proved the functionality of the platform to self-drive from one point to another with minimal human intervention.Los avances tecnológicos en Sistemas Inteligentes de Transporte (ITS) han crecido de forma exponencial durante el último siglo. El objetivo de estos avances es el de proveer de sistemas innovadores e inteligentes para ser aplicados a los diferentes medios de transporte, con el fin de conseguir un transporte mas eficiente, seguro, coordinado e inteligente. El foco de los ITS se divide principalmente en dos categorías; la primera es la mejora de los componentes ya existentes en las redes de transporte, mientras que la segunda es la de desarrollar vehículos inteligentes que hagan más fácil y eficiente el transporte. Diferentes esfuerzos de investigación se han llevado a cabo con el fin de solucionar los numerosos aspectos asociados con la conducción autónoma. Esta tesis propone una solución para la cooperación y coordinación de múltiples vehículos. Para ello, en primer lugar se desarrolló un simulador (3DCoAutoSim) de conducción basado en el motor de juegos Unity, conectado al framework Robot Operating System (ROS) y al simulador Simulation of Urban Mobility (SUMO). 3DCoAutoSim ha sido probado en diferentes condiciones y circunstancias, para posteriormente validarlo con resultados a través de varios experimentos reales controlados. Los resultados obtenidos mostraron la eficiencia del simulador para manejar diferentes situaciones, emulando los vehículos en el mundo real. En segundo lugar, se desarrolló la plataforma de investigación Intelligent Campus Automobile (iCab), que consiste en dos carritos eléctricos de golf, que fueron modificados eléctrica, mecánica y electrónicamente para darle capacidades autónomas. Cada iCab se equipó con diferentes computadoras embebidas, sensores de percepción y unidades auxiliares, con la finalidad de transformarlos en vehículos autónomos. Además, se les han dado capacidad de comunicación multimodal (V2X), se les han aplicado tres capas de control, incorporando una arquitectura de cooperación para operación en modo tren, diferentes esquemas de localización, mapeado, percepción y planificación de rutas. Innumerables experimentos han sido realizados para validar cada uno de los diferentes sistemas incorporados. Los resultados prueban la funcionalidad de esta plataforma para realizar conducción autónoma y cooperativa con mínima intervención humana.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Francisco Javier Otamendi Fernández de la Puebla.- Secretario: Hanno Hildmann.- Vocal: Pietro Cerr