The future of Earth observation in hydrology

In just the past 5 years, the field of Earth observation has progressed beyond the offerings of conventional space-agency-based platforms to include a plethora of sensing opportunities afforded by CubeSats, unmanned aerial vehicles (UAVs), and smartphone technologies that are being embraced by both for-profit companies and individual researchers. Over the previous decades, space agency efforts have brought forth well-known and immensely useful satellites such as the Landsat series and the Gravity Research and Climate Experiment (GRACE) system, with costs typically of the order of 1 billion dollars per satellite and with concept-to-launch timelines of the order of 2 decades (for new missions). More recently, the proliferation of smart-phones has helped to miniaturize sensors and energy requirements, facilitating advances in the use of CubeSats that can be launched by the dozens, while providing ultra-high (3-5 m) resolution sensing of the Earth on a daily basis. Start-up companies that did not exist a decade ago now operate more satellites in orbit than any space agency, and at costs that are a mere fraction of traditional satellite missions. With these advances come new space-borne measurements, such as real-time high-definition video for tracking air pollution, storm-cell development, flood propagation, precipitation monitoring, or even for constructing digital surfaces using structure-from-motion techniques. Closer to the surface, measurements from small unmanned drones and tethered balloons have mapped snow depths, floods, and estimated evaporation at sub-metre resolutions, pushing back on spatio-temporal constraints and delivering new process insights. At ground level, precipitation has been measured using signal attenuation between antennae mounted on cell phone towers, while the proliferation of mobile devices has enabled citizen scientists to catalogue photos of environmental conditions, estimate daily average temperatures from battery state, and sense other hydrologically important variables such as channel depths using commercially available wireless devices. Global internet access is being pursued via high-altitude balloons, solar planes, and hundreds of planned satellite launches, providing a means to exploit the "internet of things" as an entirely new measurement domain. Such global access will enable real-time collection of data from billions of smartphones or from remote research platforms. This future will produce petabytes of data that can only be accessed via cloud storage and will require new analytical approaches to interpret. The extent to which today's hydrologic models can usefully ingest such massive data volumes is unclear. Nor is it clear whether this deluge of data will be usefully exploited, either because the measurements are superfluous, inconsistent, not accurate enough, or simply because we lack the capacity to process and analyse them. What is apparent is that the tools and techniques afforded by this array of novel and game-changing sensing platforms present our community with a unique opportunity to develop new insights that advance fundamental aspects of the hydrological sciences. To accomplish this will require more than just an application of the technology: in some cases, it will demand a radical rethink on how we utilize and exploit these new observing systems

UAV Parameter Estimation with Gaussian Process Approximations

Unmanned Aerial Vehicles (UAVs) provide an alternative to manned aircraft for risk associated missions and applications where sizing constraints require miniaturized flying platforms. UAVs are currently utilised in an array of applications ranging from civilian research to military battlegrounds. A part of the development process for UAVs includes constructing a flight model. This model can be used for modern flight controller design and to develop high fidelity flight simulators. Furthermore, it also has a role in analysing stability, control and handling qualities of the platform. Developing such a model involves estimating stability and control parameters from flight data. These map the platform's control inputs to its dynamic response. The modeling process is labor intensive and requires coarse approximations. Similarly, models constructed through flight tests are only applicable to a narrow flight envelope and classical system identification approaches require prior knowledge of the model structure, which, in some instances may only be partially known. This thesis attempts to find a solution to these problems by introducing a new system identification method based on dependent Gaussian processes. The new method would allow for high fidelity non-linear flight dynamic models to be constructed through experimental data. The work is divided into two main components. The first part entails the development of an algorithm that captures cross coupling between input parameters, and learns the system stability and control derivatives. The algorithm also captures any dependencies embodied in the outputs. The second part focuses on reducing the heavy computational cost, which is a deterrent to learning the model from large test flight data sets. In addition, it explores the capabilities of the model to capture any non-stationary behavior in the aerodynamic coefficients. A modeling technique was developed that uses an additive sparse model to combine global and local Gaussian processes to learn a multi-output system. Having a combined approximation makes the model suitable for all regions of the flight envelope. In an attempt to capture the global properties, a new sampling method is introduced to gather information about the output correlations. Local properties were captured using a non-stationary covariance function with KD-trees for neighbourhood selection. This makes the model scalable to learn from high dimensional large-scale data sets. The thesis provides both theoretical underpinnings and practical applications of this approach. The theory was tested in simulation on a highly coupled oblique wing aircraft and was demonstrated on a delta-wing UAV platform using real flight data. The results were compared against an alternative parametric model and demonstrated robustness, improved identification of coupling between flight modes, sound ability to provide uncertainty estimates, and potential to be applied to a broader flight envelope

UAV Parameter Estimation with Gaussian Process Approximations

Unmanned Aerial Vehicles (UAVs) provide an alternative to manned aircraft for risk associated missions and applications where sizing constraints require miniaturized flying platforms. UAVs are currently utilised in an array of applications ranging from civilian research to military battlegrounds. A part of the development process for UAVs includes constructing a flight model. This model can be used for modern flight controller design and to develop high fidelity flight simulators. Furthermore, it also has a role in analysing stability, control and handling qualities of the platform. Developing such a model involves estimating stability and control parameters from flight data. These map the platform's control inputs to its dynamic response. The modeling process is labor intensive and requires coarse approximations. Similarly, models constructed through flight tests are only applicable to a narrow flight envelope and classical system identification approaches require prior knowledge of the model structure, which, in some instances may only be partially known. This thesis attempts to find a solution to these problems by introducing a new system identification method based on dependent Gaussian processes. The new method would allow for high fidelity non-linear flight dynamic models to be constructed through experimental data. The work is divided into two main components. The first part entails the development of an algorithm that captures cross coupling between input parameters, and learns the system stability and control derivatives. The algorithm also captures any dependencies embodied in the outputs. The second part focuses on reducing the heavy computational cost, which is a deterrent to learning the model from large test flight data sets. In addition, it explores the capabilities of the model to capture any non-stationary behavior in the aerodynamic coefficients. A modeling technique was developed that uses an additive sparse model to combine global and local Gaussian processes to learn a multi-output system. Having a combined approximation makes the model suitable for all regions of the flight envelope. In an attempt to capture the global properties, a new sampling method is introduced to gather information about the output correlations. Local properties were captured using a non-stationary covariance function with KD-trees for neighbourhood selection. This makes the model scalable to learn from high dimensional large-scale data sets. The thesis provides both theoretical underpinnings and practical applications of this approach. The theory was tested in simulation on a highly coupled oblique wing aircraft and was demonstrated on a delta-wing UAV platform using real flight data. The results were compared against an alternative parametric model and demonstrated robustness, improved identification of coupling between flight modes, sound ability to provide uncertainty estimates, and potential to be applied to a broader flight envelope

Open source R for applying machine learning to RPAS remote sensing images

The increase in the number of remote sensing platforms, ranging from satellites to close-range Remotely Piloted Aircraft System (RPAS), is leading to a growing demand for new image processing and classification tools. This article presents a comparison of the Random Forest (RF) and Support Vector Machine (SVM) machine-learning algorithms for extracting land-use classes in RPAS-derived orthomosaic using open source R packages. The camera used in this work captures the reflectance of the Red, Blue, Green and Near Infrared channels of a target. The full dataset is therefore a 4-channel raster image. The classification performance of the two methods is tested at varying sizes of training sets. The SVM and RF are evaluated using Kappa index, classification accuracy and classification error as accuracy metrics. The training sets are randomly obtained as subset of 2 to 20% of the total number of raster cells, with stratified sampling according to the land-use classes. Ten runs are done for each training set to calculate the variance in results. The control dataset consists of an independent classification obtained by photointerpretation. The validation is carried out(i) using the K-Fold cross validation, (ii) using the pixels from the validation test set, and (iii) using the pixels from the full test set. Validation with K-fold and with the validation dataset show SVM give better results, but RF prove to be more performing when training size is larger. Classification error and classification accuracy follow the trend of Kappa index

UAV-Assisted Space-Air-Ground Integrated Networks: A Technical Review of Recent Learning Algorithms

Recent technological advancements in space, air and ground components have made possible a new network paradigm called "space-air-ground integrated network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However, due to UAVs' high dynamics and complexity, the real-world deployment of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the space and terrestrial components, UAVs are expected to meet performance requirements with high flexibility and dynamics using limited resources. Therefore, employing UAVs in various usage scenarios requires well-designed planning in algorithmic approaches. In this paper, we provide a comprehensive review of recent learning-based algorithmic approaches. We consider possible reward functions and discuss the state-of-the-art algorithms for optimizing the reward functions, including Q-learning, deep Q-learning, multi-armed bandit (MAB), particle swarm optimization (PSO) and satisfaction-based learning algorithms. Unlike other survey papers, we focus on the methodological perspective of the optimization problem, which can be applicable to various UAV-assisted missions on a SAGIN using these algorithms. We simulate users and environments according to real-world scenarios and compare the learning-based and PSO-based methods in terms of throughput, load, fairness, computation time, etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional (3D) variations of these algorithms to reflect different deployment cases. Our simulation suggests that the $3$D satisfaction-based learning algorithm outperforms the other approaches for various metrics in most cases. We discuss some open challenges at the end and our findings aim to provide design guidelines for algorithm selections while optimizing the deployment of UAV-assisted SAGINs.Comment: Submitted to the IEEE Internet of Things Journal in June 202

StratoTrans : Unmanned Aerial System (UAS) 4G communication framework applied on the monitoring of road traffic and linear infrastructure

This study provides an operational solution to directly connect drones to internet by means of 4G telecommunications and exploit drone acquired data, including telemetry and imagery but focusing on video transmission. The novelty of this work is the application of 4G connection to link the drone directly to a data server where video (in this case to monitor road traffic) and imagery (in the case of linear infrastructures) are processed. However, this framework is appliable to any other monitoring purpose where the goal is to send real-time video or imagery to the headquarters where the drone data is processed, analyzed, and exploited. We describe a general framework and analyze some key points, such as the hardware to use, the data stream, and the network coverage, but also the complete resulting implementation of the applied unmanned aerial system (UAS) communication system through a Virtual Private Network (VPN) featuring a long-range telemetry high-capacity video link (up to 15 Mbps, 720 p video at 30 fps with 250 ms of latency). The application results in the real-time exploitation of the video, obtaining key information for traffic managers such as vehicle tracking, vehicle classification, speed estimation, and roundabout in-out matrices. The imagery downloads and storage is also performed thorough internet, although the Structure from Motion postprocessing is not real-time due to photogrammetric workflows. In conclusion, we describe a real-case application of drone connection to internet thorough 4G network, but it can be adapted to other applications. Although 5G will -in time- surpass 4G capacities, the described framework can enhance drone performance and facilitate paths for upgrading the connection of on-board devices to the 5G network

Using Unmanned Aerial Vehicles for Wireless Localization in Search and Rescue

This thesis presents how unmanned aerial vehicles (UAVs) can successfully assist in search and rescue (SAR) operations using wireless localization. The zone-grid to partition to capture/detect WiFi probe requests follows the concepts found in Search Theory Method. The UAV has attached a sensor, e.g., WiFi sniffer, to capture/detect the WiFi probes from victims or lost people’s smartphones. Applying the Random-Forest based machine learning algorithm, an estimation of the user\u27s location is determined with a 81.8% accuracy. UAV technology has shown limitations in the navigational performance and limited flight time. Procedures to optimize these limitations are presented. Additionally, how the UAV is maneuvered during flight is analyzed, considering different SAR flight patterns and Li-Po battery consumption rates of the UAV. Results show that controlling the UAV by remote-controll detected the most probes, but it is less power efficient compared to control it autonomously

Long-term Informative Path Planning with Autonomous Soaring

The ability of UAVs to cover large areas efficiently is valuable for information gathering missions. For long-term information gathering, a UAV may extend its endurance by accessing energy sources present in the atmosphere. Thermals are a favourable source of wind energy and thermal soaring is adopted in this thesis to enable long-term information gathering. This thesis proposes energy-constrained path planning algorithms for a gliding UAV to maximise information gain given a mission time that greatly exceeds the UAV's endurance. This thesis is motivated by the problem of probabilistic target-search performed by an energy-constrained UAV, which is tasked to simultaneously search for a lost ground target and explore for thermals to regain energy. This problem is termed informative soaring (IFS) and combines informative path planning (IPP) with energy constraints. IFS is shown to be NP-hard by showing that it has a similar problem structure to the weight-constrained shortest path problem with replenishments. While an optimal solution may not exist in polynomial time, this thesis proposes path planning algorithms based on informed tree search to find high quality plans with low computational cost. This thesis addresses complex probabilistic belief maps and three primary contributions are presented: • First, IFS is formulated as a graph search problem by observing that any feasible long-term plan must alternate between 1) information gathering between thermals and 2) replenishing energy within thermals. This is a first step to reducing the large search state space. • The second contribution is observing that a complex belief map can be viewed as a collection of information clusters and using a divide and conquer approach, cluster tree search (CTS), to efficiently find high-quality plans in the large search state space. In CTS, near-greedy tree search is used to find locally optimal plans and two global planning versions are proposed to combine local plans into a full plan. Monte Carlo simulation studies show that CTS produces similar plans to variations of exhaustive search, but runs five to 20 times faster. The more computationally efficient version, CTSDP, uses dynamic programming (DP) to optimally combine local plans. CTSDP is executed in real time on board a UAV to demonstrate computational feasibility. • The third contribution is an extension of CTS to unknown drifting thermals. A thermal exploration map is created to detect new thermals that will eventually intercept clusters, and therefore be valuable to the mission. Time windows are computed for known thermals and an optimal cluster visit schedule is formed. A tree search algorithm called CTSDrift combines CTS and thermal exploration. Using 2400 Monte Carlo simulations, CTSDrift is evaluated against a Full Knowledge method that has full knowledge of the thermal field and a Greedy method. On average, CTSDrift outperforms Greedy in one-third of trials, and achieves similar performance to Full Knowledge when environmental conditions are favourable

Robust Modular Feature-Based Terrain-Aided Visual Navigation and Mapping

The visual feature-based Terrain-Aided Navigation (TAN) system presented in this thesis addresses the problem of constraining inertial drift introduced into the location estimate of Unmanned Aerial Vehicles (UAVs) in GPS-denied environment. The presented TAN system utilises salient visual features representing semantic or human-interpretable objects (roads, forest and water boundaries) from onboard aerial imagery and associates them to a database of reference features created a-priori, through application of the same feature detection algorithms to satellite imagery. Correlation of the detected features with the reference features via a series of the robust data association steps allows a localisation solution to be achieved with a finite absolute bound precision defined by the certainty of the reference dataset. The feature-based Visual Navigation System (VNS) presented in this thesis was originally developed for a navigation application using simulated multi-year satellite image datasets. The extension of the system application into the mapping domain, in turn, has been based on the real (not simulated) flight data and imagery. In the mapping study the full potential of the system, being a versatile tool for enhancing the accuracy of the information derived from the aerial imagery has been demonstrated. Not only have the visual features, such as road networks, shorelines and water bodies, been used to obtain a position ’fix’, they have also been used in reverse for accurate mapping of vehicles detected on the roads into an inertial space with improved precision. Combined correction of the geo-coding errors and improved aircraft localisation formed a robust solution to the defense mapping application. A system of the proposed design will provide a complete independent navigation solution to an autonomous UAV and additionally give it object tracking capability