Survey on video anomaly detection in dynamic scenes with moving cameras

The increasing popularity of compact and inexpensive cameras, e.g.~dash cameras, body cameras, and cameras equipped on robots, has sparked a growing interest in detecting anomalies within dynamic scenes recorded by moving cameras. However, existing reviews primarily concentrate on Video Anomaly Detection (VAD) methods assuming static cameras. The VAD literature with moving cameras remains fragmented, lacking comprehensive reviews to date. To address this gap, we endeavor to present the first comprehensive survey on Moving Camera Video Anomaly Detection (MC-VAD). We delve into the research papers related to MC-VAD, critically assessing their limitations and highlighting associated challenges. Our exploration encompasses three application domains: security, urban transportation, and marine environments, which in turn cover six specific tasks. We compile an extensive list of 25 publicly-available datasets spanning four distinct environments: underwater, water surface, ground, and aerial. We summarize the types of anomalies these datasets correspond to or contain, and present five main categories of approaches for detecting such anomalies. Lastly, we identify future research directions and discuss novel contributions that could advance the field of MC-VAD. With this survey, we aim to offer a valuable reference for researchers and practitioners striving to develop and advance state-of-the-art MC-VAD methods.Comment: Under revie

UAV or Drones for Remote Sensing Applications in GPS/GNSS Enabled and GPS/GNSS Denied Environments

The design of novel UAV systems and the use of UAV platforms integrated with robotic sensing and imaging techniques, as well as the development of processing workflows and the capacity of ultra-high temporal and spatial resolution data, have enabled a rapid uptake of UAVs and drones across several industries and application domains.This book provides a forum for high-quality peer-reviewed papers that broaden awareness and understanding of single- and multiple-UAV developments for remote sensing applications, and associated developments in sensor technology, data processing and communications, and UAV system design and sensing capabilities in GPS-enabled and, more broadly, Global Navigation Satellite System (GNSS)-enabled and GPS/GNSS-denied environments.Contributions include:UAV-based photogrammetry, laser scanning, multispectral imaging, hyperspectral imaging, and thermal imaging;UAV sensor applications; spatial ecology; pest detection; reef; forestry; volcanology; precision agriculture wildlife species tracking; search and rescue; target tracking; atmosphere monitoring; chemical, biological, and natural disaster phenomena; fire prevention, flood prevention; volcanic monitoring; pollution monitoring; microclimates; and land use;Wildlife and target detection and recognition from UAV imagery using deep learning and machine learning techniques;UAV-based change detection

1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.Comment: MaCVi 2023 was part of WACV 2023. This report (38 pages) discusses the competition as part of MaCV

Reliable Navigational Scene Perception for Autonomous Ships in Maritime Environment

Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships

USE OF ARTIFICIAL FIDUCIAL MARKERS FOR USV SWARM COORDINATION

Typical swarm algorithms (leader-follower, artificial potentials, etc.) rely on knowledge about the pose of each vehicle and inter-vehicle proximity. This information is often obtained via Global Positioning System (GPS) and communicated via radio-frequency means.. This research examines the capabilities and limitations of using a fiducial marker system in conjunction with an artificial potential field algorithm to achieve inter-vehicle localization and coordinate the motion of unmanned surface vessels operating together in an environment where satellite and radio communications are inhibited. Using Gazebo, a physics-based robotic simulation environment, a virtual model is developed for incorporating fiducial markers on a group of autonomous surface vessels. A control framework using MATLAB and the Robot Operating System (ROS) is developed that integrates image processing, AprilTag fiducial marker detection, and artificial potential control algorithms. This architecture receives multiple video streams, detects AprilTags, and extracts pose information to control the forward motion and inter-vehicle spacing in a swarm of autonomous surface vessels. This control architecture is tested for a variety of trajectories and tuned so that the swarm can successfully maintain formation control.Lieutenant, United States NavyApproved for public release. Distribution is unlimited

Semantic-based adaptive mission planning for unmanned underwater vehicles

Current underwater robotic platforms rely upon waypoint-based scripted missions which are described by the operator a-priori. This renders systems incapable of reacting to the unexpected. In this thesis, we claim that the ability to autonomously adapt the decision making process is the key to facilitating the change over from human intervention to intelligent autonomy. We identify goal-based declarative mission planning as an attractive solution to autonomous adaptability because it combines autonomous decision making with higher levels of human interaction. Goal-based mission planning requires the use of abstract knowledge representation and situation awareness to link the prior knowledge provided by the operator with the information coming from the processed sensor data. To achieve this, we propose a semantic-based knowledge representation framework that allows this integration of prior and processed information among all different agents available in the platform. In order to evaluate adaptive mission planning techniques, we also introduce a novel metric which measures the proximity between plans. We demonstrate that this metric is better informed than previous metrics for measuring the adaptation process. In this thesis we implement three different approaches to goal-based mission planning in order to investigate which approach is most appropriate under different circumstances. The first approach, continuous mission planning, focusses on long-term deployment. This approach is based on a continuous re-assessment of the status of the mission environment. Using our proximity metric, we evaluated this approach and show that there is a high degree of similarity between our approach and the humanly driven adaptation, both in a known static environment and in a partially-known dynamic discoverable environment. The second, service-oriented mission planning, makes use of the semantic framework to provide autonomous mission planning for the dynamic discovery of the services published by the different agents in the system. This allows platform independence, easing the manual creation of mission plans, and robustness to changes. We show that this approach produces the same plans as the baseline which was explicitly provided with the platform configuration. The last approach, mission plan repair, handles the scenario where small changes occur in the mission environment and there are limited resources for planning. We develop and deploy a mission plan repair approach within a semantic-based autonomous planning system in a real underwater vehicle. Experiments demonstrate that the integrated system is capable of providing mission adaptation for maintaining the operability of the host platform in the face of unexpected events

Mission Assurance for Autonomous Underwater Vehicles

The ubiquity of autonomous vehicles (AVs) is all but inevitable, and AVs have made fantastic leaps in their capabilities, partly thanks to advances in artificial intelligence and machine learning (AI/ML). With these great capabilities should come great assurance that AVs will behave safely and achieve their operational goals, or mission, despite foreseen and unforeseen circumstances. AV software is highly complex, increasing the likelihood of faults. AI/ML decision making is poorly understood. And, all computer-based systems are vulnerable to malicious software and other cybersecurity threats. Eliminating or mitigating any one of these is an open research problem. AVs must handle all three, without the benefit of a human operator. This dissertation investigates several aspects of AV mission assurance, and offers solutions for test and evaluation starting early in the development cycle, a use case with which to experiment, and a methodology for iteratively improving assurance as more is learned about a mission and its specific risks. This dissertation focuses on autonomous underwater vehicles (AUVs). Each chapter explores particular aspects of AUV mission assurance and presents approaches to address them. We discuss the risks specific to AUV safety and mission assurance. We introduce the Digital Environment for Simulated Cyber Resilience Engineering, Test and Experimentation (DESCRETE) testbed that enables cost-effective AUV simulation, particularly with respect to system-level faults and attacks. We present the mission-assured AUV (MAAUV) use case, which we used to gather data on DESCRETE to improve the testbed and better understand mission assurance. We propose an iterative mission-assurance refinement analysis (IMARA) methodology for understanding system-failure impacts to mission. Applying IMARA to the MAAUV, we provide a guide for AUV and mission designers to best use limited assurance improvement and mitigation resources. Combining all these provides a comprehensive set of tools to improve AUV assurance

The SIMPSONS project: An integrated Mars transportation system

In response to the Request for Proposal (RFP) for an integrated transportation system network for an advanced Martian base, Frontier Transportation Systems (FTS) presents the results of the SIMPSONS project (Systems Integration for Mars Planetary Surface Operations Networks). The following topics are included: the project background, vehicle design, future work, conclusions, management status, and cost breakdown. The project focuses solely on the surface-to-surface transportation at an advanced Martian base