Wildfires have evolved significantly over the last decades, burning increasingly large forest areas every year. Smart cyber-physical systems like small Unmanned Air Vehicles (UAVs) can help to monitor, predict, and mitigate wildfires. In this paper, we present an approach to build control software for UAVs that allows autonomous monitoring of wildfires. Our proposal is underpinned by an ensemble of artificial intelligence techniques that include: (i) Recurrent Neural Networks (RNNs) to make local UAV predictions about how the fire will spread over its surrounding area; and (ii) Deep Reinforcement Learning (DRL) to learn policies that will optimize the operation of the UAV team.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Cámara-Moreno, Javier

Troya-Castilla, Javier

Vilchez Campillejo, Enrique

English

Repositorio Institucional Universidad de Málaga

This work is licensed under a Creative Commons “Attribution-NonCommercial-NoDerivatives 4.0 International” license.Towards Self-Adaptive Software for WildfireMonitoring with Unmanned Air VehiclesEnrique Vı́lchez1[0009−0009−7548−9641], Javier Troya1[0000−0002−1314−9694], andJavier Cámara1[0000−0001−6717−4775]ITIS Software, Universidad de Málaga, Spain{enriquev, jtroya, jcamara}@uma.esAbstract. Wildfires have evolved significantly over the last decades,burning increasingly large forest areas every year. Smart cyber-physicalsystems like small Unmanned Air Vehicles (UAVs) can help to monitor,predict, and mitigate wildfires. In this paper, we present an approach tobuild control software for UAVs that allows autonomous monitoring ofwildfires. Our proposal is underpinned by an ensemble of artificial intel-ligence techniques that include: (i) Recurrent Neural Networks (RNNs)to make local UAV predictions about how the fire will spread over itssurrounding area; and (ii) Deep Reinforcement Learning (DRL) to learnpolicies that will optimize the operation of the UAV team.Keywords: wildfire monitoring · artificial intelligence · UAVs1 IntroductionEvery year, large extensions of terrain are burned by wildfires. In Spain, 2022 hasbeen the worst in the last 15 years, with 267,939.64 hectares of forest devastated[3]. The use of human-supervised, software-controlled Small Unmanned Air Ve-hicles (UAVs) like drones is considered as a promising approach to effectivelymonitor and mitigate the impact of wildfires without the limitations of humanoperators (limited ability to handle complexity, long reaction time, etc.).There are multiple approaches that already employ techniques such as DeepReinforcement Learning on analogous applications that endow drones with au-tonomous behavior [2] [1]. However, these approaches are currently limited be-cause they: (i) do not consider key factors like smoke, which can have a re-markable disruptive effect on the monitoring task of UAVs, and (ii) are largelyreactive and not equipped to anticipate how the fire will spread in the shortterm. This second limitation is particularly relevant because it may result in sit-uations where drone behavior may either be too slow to react on time to avoiddrone damage, or too conservative (and thus suboptimal to effectively monitorareas that are relatively close to the fire).In this paper, we describe a work-in-progress that aims at overcoming suchlimitations by proposing an approach that allows: (i) defining a more realisticmodel of the wildfire propagation behavior that incorporates a smoke model; and(ii) a combination of machine learning techniques that will be used to anticipatethe forest fire spread and proactively adapt the behavior of the UAV team to it.This work is licensed under a Creative Commons “Attribution-NonCommercial-NoDerivatives 4.0 International” license.2 E. Vı́lchez et al.2 Problem descriptionThe core of our problem description is based on an analogous one for aircraft-class drones [2]. However, our work extends the environment description to con-sider smoke, and adapts the problem to the specific context of quadcopter UAVs.Wildfire modeling A forest area is represented as a matrix of N ×N cells,where each cell c ∈ C incorporates two time-dependent variables that represent:(i) the amount of burnable fuel in the cell F : C → N; and (ii) whether thecell is burning (B : C → {0, 1}). We assume a discrete notion of time, and forsimplicity, we represent the fuel value of a variable associated with a cell c attime instant t as e.g., Ft(c). When one time unit elapses, each cell can changein three ways:– If cell c is burning and there is remaining fuel (Ft(c) > 0 ∧ Bt(c) = 0), fuelis decreased according to a burning rate parameter β (consumption speed).– When Ft(c) = 0, the fire is extinguished in that cell (Bt+1(c) = 0).– When there is fuel remaining and the cell is not burning (Ft(c) > 0∧Bt(c) =0), we define a probability pι : C → [0, 1] of the cell getting ignited that isinversely proportional to the proximity to other burning cells.1A concrete example of how wildfire propagates through a grid where N = 25can be seen in Figure 1.Wind and smoke modeling Wind is modeled as the bias of a cell ignitingprobability, depending on the direction (north, east, south, west) and strength.For instance, if the wind blows north, any cell (c′) north of a burning cell c willsee its ignite probability pι(c) increased according to a wind speed parameter µ,while cells south of c will see their ignite probability decreased by µ.(a) (b) (c) (d)Fig. 1. Wildfire propagation in a 25x25 grid: (a) t = 0 (b) t = 5 (c) t = 10 (d) t = 15Smoke is modeled analogously to wind, with three variables for each cell:(i) S : C → {0, 1} indicates whether the cell contains smoke or not, (ii) H : C →N indicates smoke height, and (iii) DC : C → N is a dispelling smoke counter.1 Readers interested in details can refer to [2].This work is licensed under a Creative Commons “Attribution-NonCommercial-NoDerivatives 4.0 International” license.Title Suppressed Due to Excessive Length 3Smoke appears only if there is fire in a cell. After smoke appears over aburning cell c, it gains height each time unit by an ascension rate (Ht+1(c) =Ht(c) + Φ). When the cell stops burning, the smoke stops gaining height, andthe dispelling counter starts to discount (DCt+1(c) = DCt(c) − γ if Bt(c) =0 ∧ St(c) = 1) according to a discount rate γ. When DC(c) = 0, the smoke inthat cell disappears (S(c) = 0). Moreover, smoke can be spread over cells withor without fire. For a given cell c, there is a probability to get smoke based onthe weighted multiplication of contiguous cells smoke level.Unmanned Air Vehicles We assume that the UAV is in constant flightand can move at a constant speed measured in cells per time unit. The availableactions to a UAV are changing altitude, moving north, east, south, and west.The goal of a UAV is to maximize its covered forest area, i.e., to monitoras many cells on fire as possible. At the same time, the UAV should avoid thesmoke (which hampers its ability to effectively monitor its surroundings), as wellas getting too close to the fire to avoid damage. Globally, the goal for the UAVteam is to maximize the overall monitored forest area.3 Self-Adaptive Wildfire MonitoringWe envisage a solution to the problem described in Section 2 based on MAPE-K [?], considered to be one of the most successful approaches to engineering self-adaptive software that has at its core a set of models used to support reasoningat run time about when (analysis) and how (planning) to best adapt the system.For our solution, we intend to instantiate MAPE-K making use of componentsthat implement both RNNs and DRL to inform the analysis and planning stagesof the MAPE loop, respectively, as illustrated in Figure 3. The managed systemlayer (bottom) is the UAV’s software, which includes components to control bothsensors and actuators. The managing system layer (top) incorporates the variousstages that endow the system with autonomous, self-adaptive behavior:1. The monitor stage aggregates sensor information and provides it to the RNN,which predicts the next elements of a sequence of observations (e.g., aboutthe state of the terrain, other drones) based on previous ones, providing tothe analyze stage a prediction of how the fire will spread.2. Based on the output provided by the RNN component, the analyze stagedetermines whether the current situation demands an action from the UAV(e.g., because it may be too close to the fire). If analyze determines that thereis a need for adaptation, then the plan stage is triggered.3. When the plan stage is triggered, it employs DRL for decision-making. Con-cretely, the DRL component receives as input a local observation of thearea, and a prediction of how it will partially evolve in the next time units(obtained from the RNN component). The DRL generates a decision bycalculating the weighted reward of the inputs, together with variables con-cerning about other UAVs state and generates a plan as output, that consistsin moving the respective UAV to a new position in order to maximize themonitored fire area, which is passed on to the execute stage.This work is licensed under a Creative Commons “Attribution-NonCommercial-NoDerivatives 4.0 International” license.4 E. Vı́lchez et al.MonitorAnalyze PlanExecuteRNNs DRLManaging systemManaged systemEffectorsSensors12 34SensingActuationFig. 2. An overview of our self-adaptive wildfire monitoring approach4. Finally, the execute stage receives the plan and enacts its execution by co-ordinating the effectors embedded at the managed system layer.4 Conclusions and Future WorkWe have presented a work-in-pogress approach to build self-adaptive software forwildfire monitoring with UAVs that is supported by an ensemble of AI techniquesaimed at anticipating the evolution of the environment to optimize UAV teamoperations. In future work, we plan on moving towards more realistic scenariosand refining our RNN and DRL architectures to make them more precise.References1. Jain, P., Coogan, S.C., Subramanian, S.G., Crowley, M., Taylor, S., Flannigan,M.D.: A review of machine learning applications in wildfire science and management.Environmental Reviews 28, 478–505 (2020), https://cdnsciencepub.com/doi/10.1139/er-2020-00192. Julian, K.D., Kochenderfer, M.J.: Distributed wildfire surveillance with autonomousaircraft using deep reinforcement learning. Journal of Guidance, Control, and Dy-namics 42, 1768–1778 (10 2018), https://arxiv.org/abs/1810.04244v13. Ministerio para la transición ecológica y el reto demográfico: Avance informa-tivo de incendios forestales del 1 de enero al 31 de diciembre de 2022. (2023),https://www.miteco.gob.es/es/biodiversidad/temas/incendios-forestales/avanceinformativo31dediciembre2022_tcm30-560521.pdfAcknowledgements This work was partially funded by the Spanish Govern-ment (FEDER/Ministerio de Ciencia e Innovación–Agencia Estatal de Investi-gación) under projects TED2021-130523B-I00 and PID2021-125527NB-I00.

Towards Self-Adaptive Software for Wildfire Monitoring with Unmanned Air Vehicles.

https://riuma.uma.es/xmlui/bitstream/10630/27656/1/EV___JISBD_2023.pdf

Towards Self-Adaptive Software for Wildfire Monitoring with Unmanned Air Vehicles.

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional Universidad de Málaga