Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones

Safe navigation of drones in the presence of adversarial physical attacks from multiple pursuers is a challenging task. This paper proposes a novel approach, asynchronous multi-stage deep reinforcement learning (AMS-DRL), to train adversarial neural networks that can learn from the actions of multiple evolved pursuers and adapt quickly to their behavior, enabling the drone to avoid attacks and reach its target. Specifically, AMS-DRL evolves adversarial agents in a pursuit-evasion game where the pursuers and the evader are asynchronously trained in a bipartite graph way during multiple stages. Our approach guarantees convergence by ensuring Nash equilibrium among agents from the game-theory analysis. We evaluate our method in extensive simulations and show that it outperforms baselines with higher navigation success rates. We also analyze how parameters such as the relative maximum speed affect navigation performance. Furthermore, we have conducted physical experiments and validated the effectiveness of the trained policies in real-time flights. A success rate heatmap is introduced to elucidate how spatial geometry influences navigation outcomes. Project website: https://github.com/NTU-ICG/AMS-DRL-for-Pursuit-Evasion.Comment: Accepted by IEEE Transactions on Artificial Intelligenc

Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach

Equipping drones with target search capabilities is highly desirable for applications in disaster rescue and smart warehouse delivery systems. Multiple intelligent drones that can collaborate with each other and maneuver among obstacles show more effectiveness in accomplishing tasks in a shorter amount of time. However, carrying out collaborative target search (CTS) without prior target information is extremely challenging, especially with a visual drone swarm. In this work, we propose a novel data-efficient deep reinforcement learning (DRL) approach called adaptive curriculum embedded multistage learning (ACEMSL) to address these challenges, mainly 3-D sparse reward space exploration with limited visual perception and collaborative behavior requirements. Specifically, we decompose the CTS task into several subtasks including individual obstacle avoidance, target search, and inter-agent collaboration, and progressively train the agents with multistage learning. Meanwhile, an adaptive embedded curriculum (AEC) is designed, where the task difficulty level (TDL) can be adaptively adjusted based on the success rate (SR) achieved in training. ACEMSL allows data-efficient training and individual-team reward allocation for the visual drone swarm. Furthermore, we deploy the trained model over a real visual drone swarm and perform CTS operations without fine-tuning. Extensive simulations and real-world flight tests validate the effectiveness and generalizability of ACEMSL. The project is available at https://github.com/NTU-UAVG/CTS-visual-drone-swarm.git.Comment: Accepted by IEEE Transactions on Neural Networks and Learning System

Vision-based Learning for Drones: A Survey

Drones as advanced cyber-physical systems are undergoing a transformative shift with the advent of vision-based learning, a field that is rapidly gaining prominence due to its profound impact on drone autonomy and functionality. Different from existing task-specific surveys, this review offers a comprehensive overview of vision-based learning in drones, emphasizing its pivotal role in enhancing their operational capabilities under various scenarios. We start by elucidating the fundamental principles of vision-based learning, highlighting how it significantly improves drones' visual perception and decision-making processes. We then categorize vision-based control methods into indirect, semi-direct, and end-to-end approaches from the perception-control perspective. We further explore various applications of vision-based drones with learning capabilities, ranging from single-agent systems to more complex multi-agent and heterogeneous system scenarios, and underscore the challenges and innovations characterizing each area. Finally, we explore open questions and potential solutions, paving the way for ongoing research and development in this dynamic and rapidly evolving field. With growing large language models (LLMs) and embodied intelligence, vision-based learning for drones provides a promising but challenging road towards artificial general intelligence (AGI) in 3D physical world

Toward Building a Physical Proxy for Gas-Phase Sulfuric Acid Concentration Based on Its Budget Analysis in Polluted Yangtze River Delta, East China

Gaseous sulfuric acid (H2SO4) is a crucial precursor for secondary aerosol formation, particularly for new particle formation (NPF) that plays an essential role in the global number budget of aerosol particles and cloud condensation nuclei. Due to technology challenges, global-wide and long-term measurements of gaseous H2SO4 are currently very challenging. Empirical proxies for H2SO4 have been derived mainly based on short-term intensive campaigns. In this work, we performed comprehensive measurements of H2SO4 and related parameters in the polluted Yangtze River Delta in East China during four seasons and developed a physical proxy based on the budget analysis of gaseous H2SO4. Besides the photo-oxidation of SO2, we found that primary emissions can contribute considerably, particularly at night. Dry deposition has the potential to be a non-negligible sink, in addition to condensation onto particle surfaces. Compared with the empirical proxies, the newly developed physical proxy demonstrates extraordinary stability in all the seasons and has the potential to be widely used to improve the understanding of global NPF fundamentally.Peer reviewe

Cyber attack detection and isolation for a quadrotor UAV with modified sliding innovation sequences

Common vulnerabilities in typical intelligent cyber-physical systems such as unmanned aerial vehicles (UAVs) can be easily exploited by cyber attackers to cause serious accidents and harm. For successful UAV operations, security against cyber attacks is imperative. In this paper, we propose a modified sliding innovation sequences (MSIS) detector, based on the extended Kalman filter optimal state estimation, for a dynamic quadrotor system to detect cyber attacks inflicted on both its actuators and sensors in real time. These cyber attacks include random attacks, false data injection (FDI) attacks and denial-of-service (DoS) attacks. The MSIS detector computes the operator norm of the normalized innovation (residual) sequence within a sliding time window and triggers the alarm if the value is above the preset threshold. For a quadrotor undergoing rapid turns in a complex trajectory, the detector observes a reduced false alarm rate as compared to other state estimation-based detectors. To address the initial estimation error problem, we implement an iteration procedure to initiate and calibrate the detector. By evaluating the sample covariance of the normalized innovation sequence, the MSIS detector has the capability to isolate cyber attacks. Finally, simulation results of a quadrotor in a periodic, complex trajectory flight are provided to verify the effectiveness of the MSIS detection and isolation method.Ministry of Education (MOE)This work was supported by the Ministry of Education - Singapore, under its Academic Research Fund Tier 1 under Grant RG69/20

Optimization Method for Land Use of the Xi’an Rail Transit Station Area Based on a Multi-Objective Model

For the intensive, efficient, and sustainable utilization of land resources, it is of great significance to optimize the spatial allocation of different types of land use intensity in rail transit station areas. The current land use optimization model has some shortcomings in objective function, constraint conditions, and the solution process. In response to this, a new multi-objective optimization model for rail transit station land use was built. With station space efficiency as the starting point, the three objectives of the model optimization were the traffic volume, environment quality, and land balance of the rail transit station, and the constraint conditions were the plot ratio, environment quality, and efficiency level. Lingo was used to solve the optimal plot ratio of different types of land use intensity. Compared with the non-inferior solution of the rail transit station area multi-objective original model, the ideal plot ratio of various land uses obtained by the optimized new model was more reasonable. There was a relatively large gap between the non-inferior solutions of some original models and the actual conditions. In contrast, the optimized new model had stronger maneuverability. The deviation ranges of the two models were −0.4% to 0.9% on the residential land plot ratio adjustment index, −3.2% to 4.8% on the public land plot ratio adjustment index, and 1.1% to 1.9% on the commercial land plot ratio adjustment index. This research aimed to provide a basis and reference for the land use and planning of Xi’an rail transit station

Toward Collaborative Multitarget Search and Navigation with Attention‐Enhanced Local Observation

Collaborative multitarget search and navigation (CMTSN) is highly demanded in complex missions such as rescue and warehouse management. Traditional centralized and decentralized approaches fall short in terms of scalability and adaptability to real‐world complexities such as unknown targets and large‐scale missions. This article addresses this challenging CMTSN problem in three‐dimensional spaces, specifically for agents with local visual observation operating in obstacle‐rich environments. To overcome these challenges, this work presents the POsthumous Mix‐credit assignment with Attention (POMA) framework. POMA integrates adaptive curriculum learning and mixed individual‐group credit assignments to efficiently balance individual and group contributions in a sparse reward environment. It also leverages an attention mechanism to manage variable local observations, enhancing the framework's scalability. Extensive simulations demonstrate that POMA outperforms a variety of baseline methods. Furthermore, the trained model is deployed over a physical visual drone swarm, demonstrating the effectiveness and generalization of our approach in real‐world autonomous flight