Field Testing of a Stochastic Planner for ASV Navigation Using Satellite Images

We introduce a multi-sensor navigation system for autonomous surface vessels (ASV) intended for water-quality monitoring in freshwater lakes. Our mission planner uses satellite imagery as a prior map, formulating offline a mission-level policy for global navigation of the ASV and enabling autonomous online execution via local perception and local planning modules. A significant challenge is posed by the inconsistencies in traversability estimation between satellite images and real lakes, due to environmental effects such as wind, aquatic vegetation, shallow waters, and fluctuating water levels. Hence, we specifically modelled these traversability uncertainties as stochastic edges in a graph and optimized for a mission-level policy that minimizes the expected total travel distance. To execute the policy, we propose a modern local planner architecture that processes sensor inputs and plans paths to execute the high-level policy under uncertain traversability conditions. Our system was tested on three km-scale missions on a Northern Ontario lake, demonstrating that our GPS-, vision-, and sonar-enabled ASV system can effectively execute the mission-level policy and disambiguate the traversability of stochastic edges. Finally, we provide insights gained from practical field experience and offer several future directions to enhance the overall reliability of ASV navigation systems.Comment: 33 pages, 20 figures. Project website https://pcctp.github.io. arXiv admin note: text overlap with arXiv:2209.1186

Survey on video anomaly detection in dynamic scenes with moving cameras

The increasing popularity of compact and inexpensive cameras, e.g.~dash cameras, body cameras, and cameras equipped on robots, has sparked a growing interest in detecting anomalies within dynamic scenes recorded by moving cameras. However, existing reviews primarily concentrate on Video Anomaly Detection (VAD) methods assuming static cameras. The VAD literature with moving cameras remains fragmented, lacking comprehensive reviews to date. To address this gap, we endeavor to present the first comprehensive survey on Moving Camera Video Anomaly Detection (MC-VAD). We delve into the research papers related to MC-VAD, critically assessing their limitations and highlighting associated challenges. Our exploration encompasses three application domains: security, urban transportation, and marine environments, which in turn cover six specific tasks. We compile an extensive list of 25 publicly-available datasets spanning four distinct environments: underwater, water surface, ground, and aerial. We summarize the types of anomalies these datasets correspond to or contain, and present five main categories of approaches for detecting such anomalies. Lastly, we identify future research directions and discuss novel contributions that could advance the field of MC-VAD. With this survey, we aim to offer a valuable reference for researchers and practitioners striving to develop and advance state-of-the-art MC-VAD methods.Comment: Under revie

Morphology, lifecycles, and environmental sensitivities of tropical trimodal convection

Includes bibliographical references.2022 Fall.Convective clouds are ubiquitous in the tropics and typically follow a trimodal distribution of cumulus, congestus, and cumulonimbus clouds. Due to the crucial role each convective mode plays in tropical and global transport of heat and moisture, there has been both historical and recent interest in the characteristics, sensitivities, and lifecycles of these clouds. However, designing novel studies to further our knowledge has been challenging due to several limitations: the extensive computing resources needed to conduct modeling studies at sufficient resolution and scale to capture the trimodal distribution in detail; the lack of analysis tools which can objectively detect and track these clouds throughout their lifetime; and a need for more observational and modeling data of the tropical convective environments that produce these clouds. In this dissertation, three distinct but related studies that address these problems to advance the knowledge of our field on the morphology, lifecycles, and environmental sensitivities of tropical trimodal convection are presented. The first study examines the sensitivities of the tropical trimodal distribution and the convective environment to initial aerosol loading and low-level static stability. The Regional Atmospheric Modeling System (RAMS) configured as a Large Eddy Simulation (LES) is utilized to resolve all three modes in detail through two full diurnal cycles. Three initial static stabilities and three aerosol profiles are independently and simultaneously varied for a suite of nine simulations. This research found that (1) large aerosol loading and strong low-level static stability suppress the bulk environment and the intensity and coverage of convective clouds; (2) cloud and environmental responses to aerosol loading tend to be stronger than those from static stability; (3) the effects of aerosol and stability perturbations modulate each other substantially; (4) the deepest convection and highest dynamical intensity occur at moderate aerosol loading, rather than at low or high loading; and (5) most of the strongest feedbacks due to aerosol and stability perturbations are seen in the boundary layer (the latter being applied within the boundary layer themselves), though some are stronger above the freezing level. The second study presented seeks to further enhance an artificial intelligence analysis tool, the Tracking and Object-Based Analysis of Clouds (tobac) Python package, from both a scientific and procedural standpoint to enable a wider variety of research uses, including process-level studies of tropical trimodal convection. Scientific improvements to tobac v1.5 include an expansion of the tool from 2D to 3D analyses and the addition of a new spectral filtering tool. Procedural enhancements added include greater computational efficiency, data regridding capabilities, and treatments for processing data with singly or doubly periodic boundary conditions (PBCs). My distinct contributions to this work focused on the 2D to 3D expansion and the PBC treatment. These new capabilities are presented through figures, schematics, and discussion of the new science that tobac v1.5 facilitates, such as the analysis of large basin-scale datasets and detailed simulations of layered clouds, that would have been impossible before. Finally, the last study in this dissertation is a process-focused modeling study on the sensitivities of upscale growth of tropical trimodal convection to environmental aerosol loading. This project was enabled by the scientific and procedural improvements to tobac discussed in the second study, in particular the new abilities of tobac to detect and track objects in 3D and with model PBCs. Here, we used a subset of RAMS simulations from the first study, where only aerosol loading was changed and the upscale growth from shallow cumulus through congestus and cumulonimbus during the nighttime hours was investigated. This study revealed that moderately increasing aerosol loading enhances collision-coalescence processes in the middle of the cloud, which delays initial glaciation but promotes it later in the growth period. Greatly increasing aerosol, however, produces a cloud structure with a more extreme aspect ratio and greater entrainment aloft that rapidly loses buoyancy and vertical velocity with height, as well as exhibiting a greater amount of condensate loading towards the top of the cloud. We also found the relative timing of these processes to be especially important, with more rapid initial growth and lofting of condensate often inhibiting deeper convective growth

Multi-Label/Multi-Class Deep Learning Classification of Spatiotemporal Data

Human senses allow for the detection of simultaneous changes in our environments. An unobstructed field of view allows us to notice concurrent variations in different parts of what we are looking at. For example, when playing a video game, a player, oftentimes, needs to be aware of what is happening in the entire scene. Likewise, our hearing makes us aware of various simultaneous sounds occurring around us. Human perception can be affected by the cognitive ability of the brain and acuity of the senses. This is not a factor with machines. As long as a system is given a signal and instructed how to analyze this signal and extract useful information, it will be able to complete this task repeatedly with enough processing power. Automated and simultaneous detection of activity in machine learning requires the use of multi-labels. In order to detect concurrent occurrences spatially, the labels should represent the regions of interest for a particular application. For example, in this thesis, the regions of interest will be either different quadrants of a parking lot as captured on surveillance videos, four auscultation sites on patients\u27 lungs, or the two sides of the brain\u27s motor cortex (left and right). Since the labels, within the multi-labels, will be used to represent not only certain spatial locations but also different levels or types of occurrences, a multi-class/multi-level schema is necessary. In the first study, each label is appointed one of three levels of activity within the specific quadrant. In the second study, each label is assigned one of four different types of respiratory sounds. In the third study, each label is designated one of three different finger tapping frequencies. This novel multi-labeling/multi-class schema is one part of being able to detect useful information in the data. The other part of the process lies in the machine learning algorithm, the network model. In order to be able to capture the spatiotemporal characteristics of the data, selecting Convolutional Neural Network and Long Short Term Memory Network-based algorithms as the basis of the network is fitting. The following classifications are described in this thesis: 1. In the first study, one of three different motion densities are identified simultaneously in four quadrants of two sets of surveillance videos. Publicly available video recordings are the spatiotemporal data. 2. In the second study, one of four types of breathing sounds are classified simultaneously in four auscultation sites. The spatiotemporal data are publicly available respiratory sound recordings. 3. In the third study, one of three finger tapping rates are detected simultaneously in two regions of interest, the right and left sides of the brain\u27s motor cortex. The spatiotemporal data are fNIRS channel readings gathered during an index finger tapping experiment. Classification results are based on testing data which is not part of model training and validation. The success of the results is based on measures of Hamming Loss and Subset Accuracy as well Accuracy, F-Score, Sensitivity, and Specificity metrics. In the last study, model explanation is performed using Shapley Additive Explanation (SHAP) values and plotting them on an image-like background, a representation of the fNIRS channel layout used as data input. Overall, promising findings support the use of this approach in classifying spatiotemporal data with the interest of detecting different levels or types of occurrences simultaneously in several regions of interest

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 356)

This bibliography lists 192 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during November 1991. Subject coverage includes: aerospace medicine and psychology, life support systems and controlled environments, safety equipment, exobiology and extraterrestrial life, and flight crew behavior and performance

Ship recognition on the sea surface using aerial images taken by Uav : a deep learning approach

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesOceans are very important for mankind, because they are a very important source of food, they have a very large impact on the global environmental equilibrium, and it is over the oceans that most of the world commerce is done. Thus, maritime surveillance and monitoring, in particular identifying the ships used, is of great importance to oversee activities like fishing, marine transportation, navigation in general, illegal border encroachment, and search and rescue operations. In this thesis, we used images obtained with Unmanned Aerial Vehicles (UAVs) over the Atlantic Ocean to identify what type of ship (if any) is present in a given location. Images generated from UAV cameras suffer from camera motion, scale variability, variability in the sea surface and sun glares. Extracting information from these images is challenging and is mostly done by human operators, but advances in computer vision technology and development of deep learning techniques in recent years have made it possible to do so automatically. We used four of the state-of-art pretrained deep learning network models, namely VGG16, Xception, ResNet and InceptionResNet trained on ImageNet dataset, modified their original structure using transfer learning based fine tuning techniques and then trained them on our dataset to create new models. We managed to achieve very high accuracy (99.6 to 99.9% correct classifications) when classifying the ships that appear on the images of our dataset. With such a high success rate (albeit at the cost of high computing power), we can proceed to implement these algorithms on maritime patrol UAVs, and thus improve Maritime Situational Awareness

Advances in Object and Activity Detection in Remote Sensing Imagery

The recent revolution in deep learning has enabled considerable development in the fields of object and activity detection. Visual object detection tries to find objects of target classes with precise localisation in an image and assign each object instance a corresponding class label. At the same time, activity recognition aims to determine the actions or activities of an agent or group of agents based on sensor or video observation data. It is a very important and challenging problem to detect, identify, track, and understand the behaviour of objects through images and videos taken by various cameras. Together, objects and their activity recognition in imaging data captured by remote sensing platforms is a highly dynamic and challenging research topic. During the last decade, there has been significant growth in the number of publications in the field of object and activity recognition. In particular, many researchers have proposed application domains to identify objects and their specific behaviours from air and spaceborne imagery. This Special Issue includes papers that explore novel and challenging topics for object and activity detection in remote sensing images and videos acquired by diverse platforms