Autonomous Drone Landings on an Unmanned Marine Vehicle using Deep Reinforcement Learning

This thesis describes with the integration of an Unmanned Surface Vehicle (USV) and an Unmanned Aerial Vehicle (UAV, also commonly known as drone) in a single Multi-Agent System (MAS). In marine robotics, the advantage offered by a MAS consists of exploiting the key features of a single robot to compensate for the shortcomings in the other. In this way, a USV can serve as the landing platform to alleviate the need for a UAV to be airborne for long periods time, whilst the latter can increase the overall environmental awareness thanks to the possibility to cover large portions of the prevailing environment with a camera (or more than one) mounted on it. There are numerous potential applications in which this system can be used, such as deployment in search and rescue missions, water and coastal monitoring, and reconnaissance and force protection, to name but a few. The theory developed is of a general nature. The landing manoeuvre has been accomplished mainly identifying, through artificial vision techniques, a fiducial marker placed on a flat surface serving as a landing platform. The raison d'etre for the thesis was to propose a new solution for autonomous landing that relies solely on onboard sensors and with minimum or no communications between the vehicles. To this end, initial work solved the problem while using only data from the cameras mounted on the in-flight drone. In the situation in which the tracking of the marker is interrupted, the current position of the USV is estimated and integrated into the control commands. The limitations of classic control theory used in this approached suggested the need for a new solution that empowered the flexibility of intelligent methods, such as fuzzy logic or artificial neural networks. The recent achievements obtained by deep reinforcement learning (DRL) techniques in end-to-end control in playing the Atari video-games suite represented a fascinating while challenging new way to see and address the landing problem. Therefore, novel architectures were designed for approximating the action-value function of a Q-learning algorithm and used to map raw input observation to high-level navigation actions. In this way, the UAV learnt how to land from high latitude without any human supervision, using only low-resolution grey-scale images and with a level of accuracy and robustness. Both the approaches have been implemented on a simulated test-bed based on Gazebo simulator and the model of the Parrot AR-Drone. The solution based on DRL was further verified experimentally using the Parrot Bebop 2 in a series of trials. The outcomes demonstrate that both these innovative methods are both feasible and practicable, not only in an outdoor marine scenario but also in indoor ones as well

Precision Landing of a Quadrotor UAV on a Moving Target Using Low-Cost Sensors

With the use of unmanned aerial vehicles (UAVs) becoming more widespread, a need for precise autonomous landings has arisen. In the maritime setting, precise autonomous landings will help to provide a safe way to recover UAVs deployed from a ship. On land, numerous applications have been proposed for UAV and unmanned ground vehicle (UGV) teams where autonomous docking is required so that the UGVs can either recover or service a UAV in the field. Current state of the art approaches to solving the problem rely on expensive inertial measurement sensors and RTK or differential GPS systems. However, such a solution is not practical for many UAV systems. A framework to perform precision landings on a moving target using low-cost sensors is proposed in this thesis. Vision from a downward facing camera is used to track a target on the landing platform and generate high quality relative pose estimates. The landing procedure consists of three stages. First, a rendezvous stage commands the quadrotor on a path to intercept the target. A target acquisition stage then ensures that the quadrotor is tracking the landing target. Finally, visual measurements of the relative pose to the landing target are used in the target tracking stage where control and estimation are performed in a body-planar frame, without the use of GPS or magnetometer measurements. A comprehensive overview of the control and estimation required to realize the three stage landing approach is presented. Critical parts of the landing framework were implemented on an AscTec Pelican testbed. The AprilTag visual fiducial system is chosen for use as the landing target. Implementation details to improve the AprilTag detection pipeline are presented. Simulated and experimen- tal results validate key portions of the landing framework. The novel relative estimation scheme is evaluated in an indoor positioning system. Tracking and landing on a moving target is demonstrated in an indoor environment. Outdoor tests also validate the target tracking performance in the presence of wind

Power quality events analysis using wavelet transform

With an increasing usage of sensitive electronic equipment, power quality studies had grown to perform power quality data analysis. Wavelet transformation technique was founded to be more appropriate to analyze the various types of power quality events.This project compares the use of various types of wavelets at different scales and levels of decomposition on analyzing real recorded Power quality (PQ) events from Skudai 22kV distribution system. Voltage sag, voltage swell and transient event have been tested. PQ events data were extracted by RPM Power Analysis Softwarebased on a standard curve called as CBEMA curve. Background and indicators in CBEMA curve were studied, hence various PQ events were able to identify and analyze. This project also applied the 1-D WPT and 1-D SWTD of Matlab wavelet toolbox for further analysis the recorded PQ events. In 1-D WPT, four (4) types of wavelets with Shannon entropy based are applied and aim to determine the most appropriate mother wavelet for better compression and analyzed the recorded data of PQ event. Compression of voltage sag and swell waveforms were carried out with low energy loss capability was verified in order to preserve the original waveform of PQ event feature for further analysis. Performance of the 1-D WPT in compression on both voltage sag and swell events is compared based on Retain Energy (RE) and Number of Zeros (NZ) in percentage for all proposed wavelets at different scales and levels of decomposition. Ratio of energy loss per percentage of zero of compressed PQ events was also demonstrated. Compression using WT was also conducted for comparison.Presence of noise in the PQ events were de-noised using different mother wavelets at level 3 by varied three (3) types of noise structure available in 1-D SWTD. Effectiveness of using 1-D SWTD for analyzing transient, voltage sag and swell events with 3 different noise structures has been demonstrated by comparing the dispersion and distributionComparison of denoised waveform from WT was also demonstrated

Precision Landing of a Quadrotor UAV on a Moving Target Using Low-Cost Sensors

With the use of unmanned aerial vehicles (UAVs) becoming more widespread, a need for precise autonomous landings has arisen. In the maritime setting, precise autonomous landings will help to provide a safe way to recover UAVs deployed from a ship. On land, numerous applications have been proposed for UAV and unmanned ground vehicle (UGV) teams where autonomous docking is required so that the UGVs can either recover or service a UAV in the field. Current state of the art approaches to solving the problem rely on expensive inertial measurement sensors and RTK or differential GPS systems. However, such a solution is not practical for many UAV systems. A framework to perform precision landings on a moving target using low-cost sensors is proposed in this thesis. Vision from a downward facing camera is used to track a target on the landing platform and generate high quality relative pose estimates. The landing procedure consists of three stages. First, a rendezvous stage commands the quadrotor on a path to intercept the target. A target acquisition stage then ensures that the quadrotor is tracking the landing target. Finally, visual measurements of the relative pose to the landing target are used in the target tracking stage where control and estimation are performed in a body-planar frame, without the use of GPS or magnetometer measurements. A comprehensive overview of the control and estimation required to realize the three stage landing approach is presented. Critical parts of the landing framework were implemented on an AscTec Pelican testbed. The AprilTag visual fiducial system is chosen for use as the landing target. Implementation details to improve the AprilTag detection pipeline are presented. Simulated and experimen- tal results validate key portions of the landing framework. The novel relative estimation scheme is evaluated in an indoor positioning system. Tracking and landing on a moving target is demonstrated in an indoor environment. Outdoor tests also validate the target tracking performance in the presence of wind

Autonomous High-Precision Landing on a Unmanned Surface Vehicle

THE MAIN GOAL OF THIS THESIS IS THE DEVELOPMENT OF AN AUTONOMOUS HIGH-PRECISION LANDING SYSTEM OF AN UAV IN AN AUTONOMOUS BOATIn this dissertation, a collaborative method for Multi Rotor Vertical Takeoff and Landing (MR-VTOL) Unmanned Aerial Vehicle (UAV)s’ autonomous landing is presented. The majority of common UAV autonomous landing systems adopt an approach in which the UAV scans the landing zone for a predetermined pattern, establishes relative positions, and uses those positions to execute the landing. These techniques have some shortcomings, such as extensive processing being carried out by the UAV itself and requires a lot of computational power. The fact that most of these techniques only work while the UAV is already flying at a low altitude, since the pattern’s elements must be plainly visible to the UAV’s camera, creates an additional issue. An RGB camera that is positioned in the landing zone and pointed up at the sky is the foundation of the methodology described throughout this dissertation. Convolutional Neural Networks and Inverse Kinematics approaches can be used to isolate and analyse the distinctive motion patterns the UAV presents because the sky is a very static and homogeneous environment. Following realtime visual analysis, a terrestrial or maritime robotic system can transmit orders to the UAV. The ultimate result is a model-free technique, or one that is not based on established patterns, that can help the UAV perform its landing manoeuvre. The method is trustworthy enough to be used independently or in conjunction with more established techniques to create a system that is more robust. The object detection neural network approach was able to detect the UAV in 91,57% of the assessed frames with a tracking error under 8%, according to experimental simulation findings derived from a dataset comprising three different films. Also created was a high-level position relative control system that makes use of the idea of an approach zone to the helipad. Every potential three-dimensional point within the zone corresponds to a UAV velocity command with a certain orientation and magnitude. The control system worked flawlessly to conduct the UAV’s landing within 6 cm of the target during testing in a simulated setting.Nesta dissertação, é apresentado um método de colaboração para a aterragem autónoma de Unmanned Aerial Vehicle (UAV)Multi Rotor Vertical Takeoff and Landing (MR-VTOL). A maioria dos sistemas de aterragem autónoma de UAV comuns adopta uma abordagem em que o UAV varre a zona de aterragem em busca de um padrão pré-determinado, estabelece posições relativas, e utiliza essas posições para executar a aterragem. Estas técnicas têm algumas deficiências, tais como o processamento extensivo a ser efectuado pelo próprio UAV e requer muita potência computacional. O facto de a maioria destas técnicas só funcionar enquanto o UAV já está a voar a baixa altitude, uma vez que os elementos do padrão devem ser claramente visíveis para a câmara do UAV, cria um problema adicional. Uma câmara RGB posicionada na zona de aterragem e apontada para o céu é a base da metodologia descrita ao longo desta dissertação. As Redes Neurais Convolucionais e as abordagens da Cinemática Inversa podem ser utilizadas para isolar e analisar os padrões de movimento distintos que o UAV apresenta, porque o céu é um ambiente muito estático e homogéneo. Após análise visual em tempo real, um sistema robótico terrestre ou marítimo pode transmitir ordens para o UAV. O resultado final é uma técnica sem modelo, ou que não se baseia em padrões estabelecidos, que pode ajudar o UAV a realizar a sua manobra de aterragem. O método é suficientemente fiável para ser utilizado independentemente ou em conjunto com técnicas mais estabelecidas para criar um sistema que seja mais robusto. A abordagem da rede neural de detecção de objectos foi capaz de detectar o UAV em 91,57% dos fotogramas avaliados com um erro de rastreio inferior a 8%, de acordo com resultados de simulação experimental derivados de um conjunto de dados composto por três filmes diferentes. Também foi criado um sistema de controlo relativo de posição de alto nível que faz uso da ideia de uma zona de aproximação ao heliporto. Cada ponto tridimensional potencial dentro da zona corresponde a um comando de velocidade do UAV com uma certa orientação e magnitude. O sistema de controlo funcionou sem falhas para conduzir a aterragem do UAV dentro de 6 cm do alvo durante os testes num cenário simulado. Traduzido com a versão gratuita do tradutor - www.DeepL.com/Translato

Revolution

Revolutio

Helicopter Autonomous Ship Landing System

This research focuses on developing a helicopter autonomous ship landing algorithm based on the real helicopter ship landing procedure which is already proven and currently used by Navy pilots. It encompasses the entire ship landing procedure from approach to landing using a pilot-inspired vision-based navigation system. The present thesis focuses on the first step towards achieving this overarching objective, which involves modeling the flight dynamics and control of a helicopter and some preliminary simulations of a UH-60 (Blackhawk) helicopter landing on a ship. The airframe of the helicopter is modeled as a rigid body along with rotating articulated blades that can undergo flap, lag and pitch motions about the root. A UH-60 helicopter is used for a representative model due to its ample simulation and flight test data. Modeling a UH-60 helicopter is based on Blade Element Momentum Theory (BEMT), rotor aerodynamics with the Pitt-Peters linear inflow model, empennage aerodynamics and rigid body dynamics for fuselage. For the blade dynamics, the cyclic (1/rev) and collective pitch motions are prescribed and the blade (1/rev) flap and lag motions are obtained as a response to the aerodynamic and inertial forces. The helicopter control inputs and translational and attitude dynamics obtained from the model are validated with flight test data at various speeds and attitude. A linearized model is extracted based on a first-order Taylor series expansion of the nonlinear system about an equilibrium point for the purpose of determining the stability of the dynamic system, investigating sensitivity to gusts, and designing a model-based flight control system. Combined vision-based navigation and Linear Quadratic Regulator (LQR) for set-point tracking is used for disturbance rejection and tracking states. A rotatable camera is used for identifying the relative position of the helicopter with respect to the ship. Based on the position, a corresponding trajectory is computed. Considering the trade-off between transient responses and control efforts, gains for the LQR controller are chosen carefully and realistically. A fully autonomous flight is simulated from approach to landing on a ship. It consists of initial descent, steady forward flight, steady coordinated turn, deceleration, and final landing. Corresponding to each maneuver, relevant linearized model is used and gains are tuned. By using X-plane flight simulator program, the simulation data which include fuselage attitude and position at each time step are visualized with a single flight deck ship. This method allows an aircraft to land on a ship autonomously while maintaining high level of safety and accuracy without the need to capture the ship deck motions, however, by using a camera, and any other additional sensors, which will provide the accurate location of the ship relative to the helicopter. This method is not only relevant for a particular helicopter, but for any types of VTOL aircraft, manned or unmanned. Hence, it can improve the level of safety by preventing human errors that may occur during landing on a ship

Helicopter Autonomous Ship Landing System

This research focuses on developing a helicopter autonomous ship landing algorithm based on the real helicopter ship landing procedure which is already proven and currently used by Navy pilots. It encompasses the entire ship landing procedure from approach to landing using a pilot-inspired vision-based navigation system. The present thesis focuses on the first step towards achieving this overarching objective, which involves modeling the flight dynamics and control of a helicopter and some preliminary simulations of a UH-60 (Blackhawk) helicopter landing on a ship. The airframe of the helicopter is modeled as a rigid body along with rotating articulated blades that can undergo flap, lag and pitch motions about the root. A UH-60 helicopter is used for a representative model due to its ample simulation and flight test data. Modeling a UH-60 helicopter is based on Blade Element Momentum Theory (BEMT), rotor aerodynamics with the Pitt-Peters linear inflow model, empennage aerodynamics and rigid body dynamics for fuselage. For the blade dynamics, the cyclic (1/rev) and collective pitch motions are prescribed and the blade (1/rev) flap and lag motions are obtained as a response to the aerodynamic and inertial forces. The helicopter control inputs and translational and attitude dynamics obtained from the model are validated with flight test data at various speeds and attitude. A linearized model is extracted based on a first-order Taylor series expansion of the nonlinear system about an equilibrium point for the purpose of determining the stability of the dynamic system, investigating sensitivity to gusts, and designing a model-based flight control system. Combined vision-based navigation and Linear Quadratic Regulator (LQR) for set-point tracking is used for disturbance rejection and tracking states. A rotatable camera is used for identifying the relative position of the helicopter with respect to the ship. Based on the position, a corresponding trajectory is computed. Considering the trade-off between transient responses and control efforts, gains for the LQR controller are chosen carefully and realistically. A fully autonomous flight is simulated from approach to landing on a ship. It consists of initial descent, steady forward flight, steady coordinated turn, deceleration, and final landing. Corresponding to each maneuver, relevant linearized model is used and gains are tuned. By using X-plane flight simulator program, the simulation data which include fuselage attitude and position at each time step are visualized with a single flight deck ship. This method allows an aircraft to land on a ship autonomously while maintaining high level of safety and accuracy without the need to capture the ship deck motions, however, by using a camera, and any other additional sensors, which will provide the accurate location of the ship relative to the helicopter. This method is not only relevant for a particular helicopter, but for any types of VTOL aircraft, manned or unmanned. Hence, it can improve the level of safety by preventing human errors that may occur during landing on a ship