UAVs and Machine Learning Revolutionising Invasive Grass and Vegetation Surveys in Remote Arid Lands

This is the final version of the article. Available from MDPI via the DOI in this record.The monitoring of invasive grasses and vegetation in remote areas is challenging, costly, and on the ground sometimes dangerous. Satellite and manned aircraft surveys can assist but their use may be limited due to the ground sampling resolution or cloud cover. Straightforward and accurate surveillance methods are needed to quantify rates of grass invasion, offer appropriate vegetation tracking reports, and apply optimal control methods. This paper presents a pipeline process to detect and generate a pixel-wise segmentation of invasive grasses, using buffel grass (Cenchrus ciliaris) and spinifex (Triodia sp.) as examples. The process integrates unmanned aerial vehicles (UAVs) also commonly known as drones, high-resolution red, green, blue colour model (RGB) cameras, and a data processing approach based on machine learning algorithms. The methods are illustrated with data acquired in Cape Range National Park, Western Australia (WA), Australia, orthorectified in Agisoft Photoscan Pro, and processed in Python programming language, scikit-learn, and eXtreme Gradient Boosting (XGBoost) libraries. In total, 342,626 samples were extracted from the obtained data set and labelled into six classes. Segmentation results provided an individual detection rate of 97% for buffel grass and 96% for spinifex, with a global multiclass pixel-wise detection rate of 97%. Obtained results were robust against illumination changes, object rotation, occlusion, background cluttering, and floral density variation.This work was funded by the Plant Biosecurity Cooperative Research Centre (PBCRC) 2164 project, the Agriculture Victoria Research and the Queensland University of Technology (QUT). The authors would like to acknowledge Derek Sandow andWA Parks andWildlife Service for the logistic support and permits to access the survey areas at Cape Range National Park. The authors would also like to acknowledge Eduard Puig-Garcia for his contributions in co-planning the experimentation phase. The authors gratefully acknowledge the support of the QUT Research Engineering Facility (REF) Operations Team (Dirk Lessner, Dean Gilligan, Gavin Broadbent and Dmitry Bratanov), who operated the DJI S800 EVO UAV and image sensors, and performed ground referencing. We thank Gavin Broadbent for the design, manufacturing, and tuning of a two-axis gimbal for the camera. We also acknowledge the High-Performance Computing and Research Support Group at QUT, for the computational resources and services used in this work

FIREFLY: an autonomous drone for detection and monitoring of fires in rural areas / FIREFLY: um drone autônomo para detecção e monitoramento de incêndio em áreas rurais

The drones that were initially used for military purposes have expanded their use in other areas. In this work the construction of a low cost drone is presented, which was applied to the monitoring and detection of fires in rural areas. A smoke sensor MQ-2 and a high-resolution camera was used for fire monitoring and detection. The GSM technology was used for communication between the use and the drone via SMS and there is a smartphone application where the user can configure the drone flight route and calibrate the sensors within it. The proposed drone proved to be efficient in the initial tests, but showed stability problems when the number of satellites was below 4 units identified or in the presence of strong winds. The MQ-2 smoke sensor proved to be efficient in collecting environmental and functional data in fire detection. The camera can get images every 5 seconds and help monitor the area of interest. In future works, we intend to explore images to improve fire detection and classification using the technology of image processing.  The drones that were initially used for military purposes have expanded their use in other areas. In this work the construction of a low cost drone is presented, which was applied to the monitoring and detection of fires in rural areas. A smoke sensor MQ-2 and a high-resolution camera was used for fire monitoring and detection. The GSM technology was used for communication between the use and the drone via SMS and there is a smartphone application where the user can configure the drone flight route and calibrate the sensors within it. The proposed drone proved to be efficient in the initial tests, but showed stability problems when the number of satellites was below 4 units identified or in the presence of strong winds. The MQ-2 smoke sensor proved to be efficient in collecting environmental and functional data in fire detection. The camera can get images every 5 seconds and help monitor the area of interest. In future works, we intend to explore images to improve fire detection and classification using the technology of image processing. 

Unmanned Aerial Systems for Wildland and Forest Fires

Wildfires represent an important natural risk causing economic losses, human death and important environmental damage. In recent years, we witness an increase in fire intensity and frequency. Research has been conducted towards the development of dedicated solutions for wildland and forest fire assistance and fighting. Systems were proposed for the remote detection and tracking of fires. These systems have shown improvements in the area of efficient data collection and fire characterization within small scale environments. However, wildfires cover large areas making some of the proposed ground-based systems unsuitable for optimal coverage. To tackle this limitation, Unmanned Aerial Systems (UAS) were proposed. UAS have proven to be useful due to their maneuverability, allowing for the implementation of remote sensing, allocation strategies and task planning. They can provide a low-cost alternative for the prevention, detection and real-time support of firefighting. In this paper we review previous work related to the use of UAS in wildfires. Onboard sensor instruments, fire perception algorithms and coordination strategies are considered. In addition, we present some of the recent frameworks proposing the use of both aerial vehicles and Unmanned Ground Vehicles (UV) for a more efficient wildland firefighting strategy at a larger scale.Comment: A recent published version of this paper is available at: https://doi.org/10.3390/drones501001

Mogućnosti primjene bezpilotne letjelice u zaštiti šuma i požara

Bespilotne letjelice pružaju jednostavniji pristup informacijama na daljinu, nije ovisno o snimanju satelitskih intervala, cijena je prihvatljivija i repeticije mogu biti svakodnevne. U ovom radu obrađivane su podjele bespilotnih letjelica, njihove namjene i pravni propisi korištenja. Također na primjerima u monitoringu različitih staništa, pogotovo u zaštiti šuma općenito i protupožarno opisane su mogućnosti primjene. Analizirajući mogućnosti uočena su i ograničenja kod kartiranja prostora, koordinacija sa žurnim službama i slično. Zaključno je iskazna opća korist koja s vremenom može prerast u nužnost i kvalitetu potvrde terenskih podataka pri donošenju odluka. Također netreba se zanositi da stroj može zamjeniti ljuski faktor na terenu.The UAV‵s are giving us more simple acces to long distance informations, they do not depend of recording of satellite waves, the price is more acceptable and there is a possibility of daily repetitions. In this thesis it is handled the divisions of UAV‵s, it‵s appliances and legal regulations of their usage. There are also described various possibilities of appliance in monitoring of different habitats, especially in protection of forests in general an firefighting. By analysing possibilities there have been noticed restrictions in mapping of the area, coordination with emergency services and other similar problems. To conclude is that it is pointed out the general benefit which in time can become a necessity and also a quality of confirmation of field data in making decisions . It also should not be absorbed in idea that on fild a machine can replace human factor

Thermal infrared video stabilization for aerial monitoring of active wildfires

Measuring wildland fire behavior is essential for fire science and fire management. Aerial thermal infrared (TIR) imaging provides outstanding opportunities to acquire such information remotely. Variables such as fire rate of spread (ROS), fire radiative power (FRP), and fireline intensity may be measured explicitly both in time and space, providing the necessary data to study the response of fire behavior to weather, vegetation, topography, and firefighting efforts. However, raw TIR imagery acquired by unmanned aerial vehicles (UAVs) requires stabilization and georeferencing before any other processing can be performed. Aerial video usually suffers from instabilities produced by sensor movement. This problem is especially acute near an active wildfire due to fire-generated turbulence. Furthermore, the nature of fire TIR video presents some specific challenges that hinder robust interframe registration. Therefore, this article presents a software-based video stabilization algorithm specifically designed for TIR imagery of forest fires. After a comparative analysis of existing image registration algorithms, the KAZE feature-matching method was selected and accompanied by pre- and postprocessing modules. These included foreground histogram equalization and a multireference framework designed to increase the algorithm's robustness in the presence of missing or faulty frames. The performance of the proposed algorithm was validated in a total of nine video sequences acquired during field fire experiments. The proposed algorithm yielded a registration accuracy between 10 and 1000x higher than other tested methods, returned 10x more meaningful feature matches, and proved robust in the presence of faulty video frames. The ability to automatically cancel camera movement for every frame in a video sequence solves a key limitation in data processing pipelines and opens the door to a number of systematic fire behavior experimental analyses. Moreover, a completely automated process supports the development of decision support tools that can operate in real time during an emergency

Enhancing Wildfire Propagation Model Predictions Using Aerial Swarm-Based Real-Time Wind Measurements:A Conceptual Framework

The dynamic behaviour of wildfires is mainly influenced by weather, fuel, and topography. Based on fundamental conservation laws involving numerous physical processes and large scales, atmospheric models require substantial computational resources. Therefore, coupling wildfire and atmospheric models is impractical for high resolutions. Instead, a static atmospheric wind field is typically input into the wildfire model, either as boundary conditions on the control surface or distributed over the control volume. Wildfire propagation models may be (i) data-driven; theoretical; or mechanistic surrogates. Data-driven models are beyond the scope of this paper. Theoretical models are based on conservation laws (species, energy, mass, momentum) and are, therefore, computationally intensive; e.g. the Fire Dynamics Simulator (FDS). Mechanistic surrogate models do not closely follow fire dynamics laws, but related laws observed to make predictions more efficiently with sufficient accuracy; e.g. FARSITE, and FDS with the Level Set model (FDS-LS). Whether theoretical or mechanistic surrogate, these wildfire models may be coupled with or decoupled from wind models (e.g. Navier-Stokes equations). Only coupled models account for the effect of the fire on the wind field. In this paper, a series of simulations of wildfire propagation on grassland are performed using FDS-LS to study the impact of the fire-induced wind on the fire propagation dynamics. Results show that coupling leads to higher Rates of Spread (RoS), closer to those reported from field experiments, with increasing wind speeds and higher terrain slopes strengthening this trend. Aiming to capture the fire–wind interaction without the hefty cost of solving Navier-Stokes equations, a conceptual framework is proposed: 1) a swarm of unmanned aerial vehicles measure wind velocities at flight height; 2) the wind field is constructed with the acquired data; 3) the high-altitude wind field is mapped to near-surface, and 4) the near-surface wind field is fed into the wildfire model periodically. A series of simulations are performed using an in-house decoupled physics-based reduced-order fire propagation model (FireProM-F) enhanced by wind field “measurements”. In this proof of concept, wind velocities are not measured but extracted from physics-based Large Eddy Simulations taken as ground truth. Unsurprisingly, higher measurement frequencies lead to more accurate and realistic predictions of the propagating fire front. An initial attempt is made to study the effect of wind measurement uncertainty on the model predictions by adding Gaussian noise. Preliminary results show that higher noise leads to the fire front displaying more irregular shapes and slower propagation

Early wildfire detection by air quality sensors on unmanned aerial vehicles: Optimization and feasibility

“Millions of acres of forests are destroyed by wildfires every year, causing ecological, environmental, and economical losses. The recent wildfires in Australia and the Western U.S. smothered multiple states with more than fifty million acres charred by the blazes. The warmer and drier climate makes scientists expect increases in the severity and frequency of wildfires and the associated risks in the future. These inescapable crises highlight the urgent need for early detection and prevention of wildfires. This work proposed an energy management framework that integrated unmanned aerial vehicle (UAV) with air quality sensors for early wildfire detection and forest monitoring. An autonomous patrol solution that effectively detects wildfire events, while preserving the UAV battery for a larger area of coverage was developed. The UAV can send real-time data (e.g., sensor readings, thermal pictures, videos, etc) to nearby communications base stations (BSs) when a wildfire is detected. An optimization problem that minimized the total UAV’s consumed energy and satisfied a certain quality-of-service (QoS) data rate were formulated and solved. More specifically, this study optimized the flight track of a UAV and the transmit power between the UAV and BSs. Finally, selected simulation results that illustrate the advantages of the proposed model were proposed”--Abstract, page iii

Assessment of hazard tree/snag detection using drone-based, multi-spectral sensors

Snags are an integral component of forest ecosystems as they provide habitat for a number of different species and add complexity to vertical forest structure. However, snags also may pose as potential hazards to people and property. Efficient and effective methods to locate and assess snags/hazard trees holds value to resource and conservation managers. This study aimed to assess the feasibility of using drone-based, multi-spectral sensors for detecting snags/hazard trees. The methods used in the study included an autonomous drone flight over the study areas, orthomosaic processing, object-based image analysis (OBIA), an accuracy assessment, and a field ground truth. The results provided sufficient evidence of drone-based, multi-spectral sensors being effective at detecting snags/hazard trees. However, the methods used in this study were found to only be accurate at detecting high quality/hazard snags. Segmentation parameters had a significant impact on the degree of quality/hazard of snag that the algorithm could detect. The orthomosaic classification was considered as highly accurate with an overall accuracy of 93.4%. Resource and conservation managers can effectively use the methods from this study for a variety of applications that aim to promote biodiversity and/or minimize public hazards