A Multiagent Deep Reinforcement Learning Approach for Path Planning in Autonomous Surface Vehicles: The Ypacaraí Lake Patrolling Case

Article number 9330612Autonomous surfaces vehicles (ASVs) excel at monitoring and measuring aquatic nutrients due to their autonomy, mobility, and relatively low cost. When planning paths for such vehicles, the task of patrolling with multiple agents is usually addressed with heuristics approaches, such as Reinforcement Learning (RL), because of the complexity and high dimensionality of the problem. Not only do efficient paths have to be designed, but addressing disturbances in movement or the battery’s performance is mandatory. For this multiagent patrolling task, the proposed approach is based on a centralized Convolutional Deep Q-Network, designed with a final independent dense layer for every agent to deal with scalability, with the hypothesis/assumption that every agent has the same properties and capabilities. For this purpose, a tailored reward function is created which penalizes illegal actions (such as collisions) and rewards visiting idle cells (cells that remains unvisited for a long time). A comparison with various multiagent Reinforcement Learning (MARL) algorithms has been done (Independent Q-Learning, Dueling Q-Network and multiagent Double Deep Q-Learning) in a case-study scenario like the Ypacaraí lake in Asunción (Paraguay). The training results in multiagent policy leads to an average improvement of 15% compared to lawn mower trajectories and a 6% improvement over the IDQL for the case-study considered. When evaluating the training speed, the proposed approach runs three times faster than the independent algorithm.Ministerio de Ciencia, Innovación y Universidades (España) RTI2018-098964-B-I00Junta de Andalucía(España) PY18-RE000

Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learning

Coverage path planning (CPP) is a critical problem in robotics, where the goal is to find an efficient path that covers every point in an area of interest. This work addresses the power-constrained CPP problem with recharge for battery-limited unmanned aerial vehicles (UAVs). In this problem, a notable challenge emerges from integrating recharge journeys into the overall coverage strategy, highlighting the intricate task of making strategic, long-term decisions. We propose a novel proximal policy optimization (PPO)-based deep reinforcement learning (DRL) approach with map-based observations, utilizing action masking and discount factor scheduling to optimize coverage trajectories over the entire mission horizon. We further provide the agent with a position history to handle emergent state loops caused by the recharge capability. Our approach outperforms a baseline heuristic, generalizes to different target zones and maps, with limited generalization to unseen maps. We offer valuable insights into DRL algorithm design for long-horizon problems and provide a publicly available software framework for the CPP problem.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Preparing for Future Forest Fires: Emerging Technologies and Innovations

Forest fires are part of the global ecosystems occurring for a long time in earth history.  These forest fires are part of the processes which establish the ecosystems and directly influence plant species composition within the ecosystems. However, the anthropogenic effect has changed this relationship causing an increasing number of forest fires Human activities have also changed world climate and future climate is expected to increase in temperature with dire consequences on the earth environment. These changes will profoundly impact on the earth’s socio-economic and human well-being. One of the effects of higher global temperature is increasing forest fires occurrences with stronger intensities.  There is a need to develop innovation and new technologies to manage these future fires. This paper aims to review various innovations and new technologies that can be used for the whole spectrum of forest fire management, from forest fire prediction to forest restoration of burnt areas. Emerging technologies such as geospatial technologies, the Internet of Things (IoT), Artificial Intelligence, 5G & enhanced connectivity, the Internet of Behaviors (IoB), virtual and augmented reality, and robotics are discussed and potential applications to forest fire management are discussed. Adaptation of these technologies is vital in the effective management of future forest fires. Key words: Climate Change, Future Fires, InnovationsKebakaran hutan merupakan bagian dari ekosistem global yang terjadi sejak lama dalam sejarah bumi. Kebakaran hutan ini merupakan bagian dari proses yang membentuk ekosistem dan secara langsung mempengaruhi komposisi spesies tumbuhan di dalam ekosistem. Namun, efek antropogenik telah mengubah hubungan ini yang menyebabkan peningkatan jumlah kebakaran hutan Aktivitas manusia juga telah mengubah iklim dunia dan iklim di masa depan diperkirakan akan meningkatkan suhu dengan konsekuensi yang mengerikan pada lingkungan bumi. Perubahan ini akan sangat berdampak pada sosial ekonomi bumi dan kesejahteraan manusia. Salah satu dampak dari peningkatan suhu global adalah meningkatnya kejadian kebakaran hutan dengan intensitas yang lebih kuat. Ada kebutuhan untuk mengembangkan inovasi dan teknologi baru untuk mengelola kebakaran di masa depan ini. Tulisan ini bertujuan untuk mengkaji berbagai inovasi dan teknologi baru yang dapat digunakan untuk seluruh spektrum penanggulangan kebakaran hutan, mulai dari prediksi kebakaran hutan hingga restorasi hutan pada kawasan yang terbakar. Teknologi yang muncul seperti teknologi geospasial, Internet of Things (IoT), Artificial Intelligence, 5G & konektivitas yang ditingkatkan, Internet of Behaviors (IoB), virtual dan augmented reality, dan robotika dibahas dan aplikasi potensial untuk manajemen kebakaran hutan dibahas. Adaptasi teknologi ini sangat penting dalam pengelolaan kebakaran hutan yang efektif di masa depan. Kata kunci: Perubahan Iklim, Kebakaran di Masa Depan, Inovas

MODELING OF INNOVATIVE LIGHTER-THAN-AIR UAV FOR LOGISTICS, SURVEILLANCE AND RESCUE OPERATIONS

An unmanned aerial vehicle (UAV) is an aircraft that can operate without the presence of pilots, either through remote control or automated systems. The first part of the dissertation provides an overview of the various types of UAVs and their design features. The second section delves into specific experiences using UAVs as part of an automated monitoring system to identify potential problems such as pipeline leaks or equipment damage by conducting airborne surveys.Lighter-than-air UAVs, such as airships, can be used for various applications, from aerial photography, including surveying terrain, monitoring an area for security purposes and gathering information about weather patterns to surveillance. The third part reveals the applications of UAVs for assisting in search and rescue operations in disaster situations and transporting natural gas. Using PowerSim software, a model of airship behaviour was created to analyze the sprint-and-drift concept and study methods of increasing the operational time of airships while having a lower environmental impact when compared to a constantly switched-on engine. The analysis provided a reliable percentage of finding the victim during patrolling operations, although it did not account for victim behaviour. The study has also shown that airships may serve as a viable alternative to pipeline transportation for natural gas. The technology has the potential to revolutionize natural gas transportation, optimizing efficiency and reducing environmental impact. Additionally, airships have a unique advantage in accessing remote and otherwise inaccessible areas, providing significant benefits in the energy sector. The employment of this technology was studied to be effective in specific scenarios, and it will be worth continuing to study it for a positive impact on society and the environment