Coordination and navigation of heterogeneous MAV-UGV formations localized by a 'hawk-eye'-like approach under a model predictive control scheme

n approach for coordination and control of 3D heterogeneous formations of unmanned aerial and ground vehicles under hawk-eye-like relative localization is presented in this paper. The core of the method lies in the use of visual top-view feedback from flying robots for the stabilization of the entire group in a leader–follower formation. We formulate a novel model predictive control-based methodology for guiding the formation. The method is employed to solve the trajectory planning and control of a virtual leader into a desired target region. In addition, the method is used for keeping the following vehicles in the desired shape of the group. The approach is designed to ensure direct visibility between aerial and ground vehicles, which is crucial for the formation stabilization using the hawk-eye-like approach. The presented system is verified in numerous experiments inspired by search-and-rescue applications, where the formation acts as a searching phalanx. In addition, stability and convergence analyses are provided to explicitly determine the limitations of the method in real-world applications

Dronecrypt - An Efficient Cryptographic Framework for Small Aerial Drones

Aerial drones are becoming an integral part of application domains including but not limited to, military operations, package delivery, construction, monitoring and search/rescue operations. It is critical to ensure the cyber security of networked aerial drone systems in these applications. Standard cryptographic services can be deployed to provide basic security services; however, they have been shown to be inefficient in terms of energy and time consumption, especially for small aerial drones with resource-limited processors. Therefore, there is a significant need for an efficient cryptographic framework that can meet the requirements of small aerial drones. We propose an improved cryptographic framework for small aerial drones, which offers significant energy efficiency and speed advantages over standard cryptographic techniques. (i) We create (to the best of our knowledge) the first optimized public key infrastructure (PKI) based framework for small aerial drones, which provides energy efficient techniques by harnessing special precomputation methods and optimized elliptic curves. (ii) We also integrate recent light-weight symmetric primitives into our PKI techniques to provide a full-fledged cryptographic framework. (iii) We implemented standard counterparts and our proposed techniques on an actual small aerial drone (Crazyflie 2.0), and provided an in-depth energy analysis. Our experiments showed that our improved cryptographic framework achieves up to 35$\times$ lower energy consumption than its standard counterpart

NAVIGATION AND AUTONOMOUS CONTROL OF MAVS IN GPS-DENIED ENVIRONMENTS

Ph.DDOCTOR OF PHILOSOPH

Drone hacking with Raspberry-Pi 3 and WiFi Pineapple: security and privacy threats for the internet-of-things

The use of Internet-of-Things (IoT) technology is growing exponentially as more consumers and businesses acknowledge the benefits offered by the intelligent and smart devices. Drone technology is a rapidly emerging sector within the IoT and the risk of hacking could not only cause a data breach, it could also pose a major risk to the public safety. Thanks to their versatile applications and access to real-time data, commercial drones are used across a wide variety of smart city applications. However, as with many IoT devices, security is often an afterthought, leaving many drones vulnerable to hackers. This paper investigates the current state of drone security and demonstrates a set of WiFi enabled drone vulnerabilities. Five different types of attacks, together with the potential of automation of attacks, was identified and applied to two different types of commercially available drones. The communication links are investigated for the attacks, i.e. Denial of Service, Deauthentication Methods, Man-in-the-Middle, Unauthorised Root Access and Packet Spoofing. Lastly, the unauthorised root access was automated through the use of a Raspberry-Pi 3 and WiFi Pineapple. Furthermore, we outlined the methodology for each attack, and the experimental part outlines the findings and processes of the attacks. Finally, the paper addresses the current state of drone security, management, control, resilience, security, and privacy concerns

UAV or Drones for Remote Sensing Applications in GPS/GNSS Enabled and GPS/GNSS Denied Environments

The design of novel UAV systems and the use of UAV platforms integrated with robotic sensing and imaging techniques, as well as the development of processing workflows and the capacity of ultra-high temporal and spatial resolution data, have enabled a rapid uptake of UAVs and drones across several industries and application domains.This book provides a forum for high-quality peer-reviewed papers that broaden awareness and understanding of single- and multiple-UAV developments for remote sensing applications, and associated developments in sensor technology, data processing and communications, and UAV system design and sensing capabilities in GPS-enabled and, more broadly, Global Navigation Satellite System (GNSS)-enabled and GPS/GNSS-denied environments.Contributions include:UAV-based photogrammetry, laser scanning, multispectral imaging, hyperspectral imaging, and thermal imaging;UAV sensor applications; spatial ecology; pest detection; reef; forestry; volcanology; precision agriculture wildlife species tracking; search and rescue; target tracking; atmosphere monitoring; chemical, biological, and natural disaster phenomena; fire prevention, flood prevention; volcanic monitoring; pollution monitoring; microclimates; and land use;Wildlife and target detection and recognition from UAV imagery using deep learning and machine learning techniques;UAV-based change detection

Reference Model for Interoperability of Autonomous Systems

This thesis proposes a reference model to describe the components of an Un-manned Air, Ground, Surface, or Underwater System (UxS), and the use of a single Interoperability Building Block to command, control, and get feedback from such vehicles. The importance and advantages of such a reference model, with a standard nomenclature and taxonomy, is shown. We overview the concepts of interoperability and some efforts to achieve common refer-ence models in other areas. We then present an overview of existing un-manned systems, their history, characteristics, classification, and missions. The concept of Interoperability Building Blocks (IBB) is introduced to describe standards, protocols, data models, and frameworks, and a large set of these are analyzed. A new and powerful reference model for UxS, named RAMP, is proposed, that describes the various components that a UxS may have. It is a hierarchical model with four levels, that describes the vehicle components, the datalink, and the ground segment. The reference model is validated by showing how it can be applied in various projects the author worked on. An example is given on how a single standard was capable of controlling a set of heterogeneous UAVs, USVs, and UGVs

Computationally-efficient visual inertial odometry for autonomous vehicle

This thesis presents the design, implementation, and validation of a novel nonlinearfiltering based Visual Inertial Odometry (VIO) framework for robotic navigation in GPSdenied environments. The system attempts to track the vehicle’s ego-motion at each time instant while capturing the benefits of both the camera information and the Inertial Measurement Unit (IMU). VIO demands considerable computational resources and processing time, and this makes the hardware implementation quite challenging for micro- and nanorobotic systems. In many cases, the VIO process selects a small subset of tracked features to reduce the computational cost. VIO estimation also suffers from the inevitable accumulation of error. This limitation makes the estimation gradually diverge and even fail to track the vehicle trajectory over long-term operation. Deploying optimization for the entire trajectory helps to minimize the accumulative errors, but increases the computational cost significantly. The VIO hardware implementation can utilize a more powerful processor and specialized hardware computing platforms, such as Field Programmable Gate Arrays, Graphics Processing Units and Application-Specific Integrated Circuits, to accelerate the execution. However, the computation still needs to perform identical computational steps with similar complexity. Processing data at a higher frequency increases energy consumption significantly. The development of advanced hardware systems is also expensive and time-consuming. Consequently, the approach of developing an efficient algorithm will be beneficial with or without hardware acceleration. The research described in this thesis proposes multiple solutions to accelerate the visual inertial odometry computation while maintaining a comparative estimation accuracy over long-term operation among state-ofthe- art algorithms. This research has resulted in three significant contributions. First, this research involved the design and validation of a novel nonlinear filtering sensor-fusion algorithm using trifocal tensor geometry and a cubature Kalman filter. The combination has handled the system nonlinearity effectively, while reducing the computational cost and system complexity significantly. Second, this research develops two solutions to address the error accumulation issue. For standalone self-localization projects, the first solution applies a local optimization procedure for the measurement update, which performs multiple corrections on a single measurement to optimize the latest filter state and covariance. For larger navigation projects, the second solution integrates VIO with additional pseudo-ranging measurements between the vehicle and multiple beacons in order to bound the accumulative errors. Third, this research develops a novel parallel-processing VIO algorithm to speed up the execution using a multi-core CPU. This allows the distribution of the filtering computation on each core to process and optimize each feature measurement update independently. The performance of the proposed visual inertial odometry framework is evaluated using publicly-available self-localization datasets, for comparison with some other open-source algorithms. The results illustrate that a proposed VIO framework is able to improve the VIO’s computational efficiency without the installation of specialized hardware computing platforms and advanced software libraries

IDENTIFICAÇÃO DE SÍTIOS DE REPRODUÇÃO DE AEDES AEGYPTI COM AERONAVE REMOTAMENTE PILOTADA (ARP)

A drone and its flight accessories are called Remotely Piloted Aircraft System (RPAS - Remotely Piloted Aircraft System), being a tool with a wide range of applications in several areas. The research explored new possibilities for the use of RPAS with a focus on the diagnosis and monitoring of breeding sites for Aedes aegypti. For this, objects considered as potential breeding grounds for mosquito larvae were distributed in environments that allowed greater or lesser visual detection of targets (packages / containers) in four environments: soil covered with dry grass, exposed soil, soil covered with low grass. and soil covered with tall grass. We use RPAS, Phantom 4 Pro with an Ipad Mini 4 mobile device and the DJI GO program for flights. We fly over targets for photographic recording at four heights from the ground (20m, 30m, 60m and 80m). The visual detection of the targets was carried out by a group of 10 people called a jury. The Jury assessed the greater or lesser probability of target detection, depending on three variables: type of target, type of environment and height of aerial photography. Photographs taken at a height of 30 meters represented the largest number of targets identified (30% of the targets). The most identified targets were tires, pet bottles, cans of beer and cans of paint. The least identified were colored plastic canisters and beer bottles. The research helped to improve operational procedures for controlling and combating endemics and epidemics, which may identify possible mosquito breeding sites through RPA, monitoring areas of difficult access that pose a risk to people's physical integrity.Um drone e seus complementos de voo são denominados Sistema de Aeronave Remotamente Pilotada (RPAS - Remotely Piloted Aircraft System), sendo uma ferramenta com ampla gama de aplicações em diversas áreas. A pesquisa prospectou novas possibilidades de uso de RPAS com enfoque no diagnóstico e monitoramento de locais de reprodução de Aedes aegypti. Para isso, objetos considerados como potenciais criadouros de larvas de mosquito foram distribuídos em ambientes que permitiam maior ou menor detecção visual dos alvos (embalagens/recipientes) em quatro ambientes: solo coberto com gramínea seca, solo exposto, solo coberto com gramínea de porte baixo e solo coberto com gramínea de porte alto. Foi utilizado RPAS, Phantom 4 Pro com dispositivo móvel e o programa nativo da RPA para os voos. Sobrevoamos alvos para registro fotográfico em quatro alturas do solo (20m, 30m, 60m e 80m). A detecção visual dos alvos foi realizada por um grupo de 10 pessoas denominado júri. O Júri aferiu a maior ou menor probabilidade de detecção de alvos, em função de três variáveis: tipo de alvo, tipo de ambiente e altura de tomada da fotografia aérea. Fotografias obtidas a 30 metros de altura representaram o maior número de alvos identificados (30% dos alvos). Os alvos mais identificados foram pneu, garrafa PET, latas de cerveja e latas de tinta. Os menos identificados foram vasilhas plásticas coloridas e garrafas de cerveja. A pesquisa colaborou para o aperfeiçoamento de procedimentos operacionais de controle e combate a endemias e epidemias, que poderão identificar possíveis criadouros do mosquito por meio de RPA, monitorando áreas de difícil acesso que ofereçam risco a integridade física das pessoas. Palavras-chave: drone; geotecnologias; arboviroses; dengue.   Identification of reproduction sites of Aedes aegypti with remote pilot aircraft (ARP)   ABSTRACT: A drone and its flight accessories are called Remotely Piloted Aircraft System (RPAS - Remotely Piloted Aircraft System), being a tool with a wide range of applications in several areas. The research explored new possibilities for the use of RPAS with a focus on the diagnosis and monitoring of breeding sites for Aedes aegypti. For this, objects considered as potential breeding grounds for mosquito larvae were distributed in environments that allowed greater or lesser visual detection of targets (packages / containers) in four environments: soil covered with dry grass, exposed soil, soil covered with low grass. and soil covered with tall grass. Was used RPAS, Phantom 4 Pro with an Ipad Mini 4 mobile device and the DJI GO program for flights. We fly over targets for photographic recording at four heights from the ground (20m, 30m, 60m and 80m). The visual detection of the targets was carried out by a group of 10 people called a jury. The Jury assessed the greater or lesser probability of target detection, depending on three variables: type of target, type of environment and height of aerial photography. Photographs taken at a height of 30 meters represented the largest number of targets identified (30% of the targets). The most identified targets were tires, pet bottles, cans of beer and cans of paint. The least identified were colored plastic canisters and beer bottles. The research helped to improve operational procedures for controlling and combating endemics and epidemics, which may identify possible mosquito breeding sites through RPA, monitoring areas of difficult access that pose a risk to people's physical integrity. Keywords: drone; geotecnologies; arbovírus; dengue

Machine learning techniques to estimate the dynamics of a slung load multirotor UAV system

This thesis addresses the question of designing robust and flexible controllers to enable autonomous operation of a multirotor UAV with an attached slung load for general cargo transport. This is achieved by following an experimental approach; real flight data from a slung load multirotor coupled system is used as experience, allowing for a computer software to estimate the pose of the slung in order to propose a swing-free controller that will dampen the oscillations of the slung load when the multirotor is following a desired flight trajectory. The thesis presents the reader with a methodology describing the development path from vehicle design and modelling over slung load state estimators to controller synthesis. Attaching a load via a cable to the underside of the aircraft alters the mass distribution of the combined "airborne entity" in a highly dynamic fashion. The load will be subject to inertial, gravitational and unsteady aerodynamic forces which are transmitted to the aircraft via the cable, providing another source of external force to the multirotor platform and thus altering the flight dynamic response characteristics of the vehicle. Similarly the load relies on the forces transmitted by the multirotor to alter its state, which is much more difficult to control. The principle research hypothesis of this thesis is that the dynamics of the coupled system can be identified by applying Machine Learning techniques. One of the major contributions of this thesis is the estimator that uses real flight data to train an unstructured black-box algorithm that can output the position vector of the load using the vehicle pose and pilot pseudo-controls as input. Experimental results show very accurate position estimation of the load using the machine learning estimator when comparing it with a motion tracking system (~2% offset). Another contribution lies in the avionics solution created for data collection, algorithm execution and control of multirotor UAVs, experimental results show successful autonomous flight with a range of algorithms and applications. Finally, to enable flight capabilities of a multirotor with slung load, a control system is developed that dampens the oscillations of the load; the controller uses a feedback approach to simultaneously prevent exciting swing and to actively dampen swing in the slung load. The methods and algorithms developed in this thesis are validated by flight testing