Iterative, Deep Synthetic Aperture Sonar Image Segmentation

Synthetic aperture sonar (SAS) systems produce high-resolution images of the seabed environment. Moreover, deep learning has demonstrated superior ability in finding robust features for automating imagery analysis. However, the success of deep learning is conditioned on having lots of labeled training data, but obtaining generous pixel-level annotations of SAS imagery is often practically infeasible. This challenge has thus far limited the adoption of deep learning methods for SAS segmentation. Algorithms exist to segment SAS imagery in an unsupervised manner, but they lack the benefit of state-of-the-art learning methods and the results present significant room for improvement. In view of the above, we propose a new iterative algorithm for unsupervised SAS image segmentation combining superpixel formation, deep learning, and traditional clustering methods. We call our method Iterative Deep Unsupervised Segmentation (IDUS). IDUS is an unsupervised learning framework that can be divided into four main steps: 1) A deep network estimates class assignments. 2) Low-level image features from the deep network are clustered into superpixels. 3) Superpixels are clustered into class assignments (which we call pseudo-labels) using $k$-means. 4) Resulting pseudo-labels are used for loss backpropagation of the deep network prediction. These four steps are performed iteratively until convergence. A comparison of IDUS to current state-of-the-art methods on a realistic benchmark dataset for SAS image segmentation demonstrates the benefits of our proposal even as the IDUS incurs a much lower computational burden during inference (actual labeling of a test image). Finally, we also develop a semi-supervised (SS) extension of IDUS called IDSS and demonstrate experimentally that it can further enhance performance while outperforming supervised alternatives that exploit the same labeled training imagery.Comment: arXiv admin note: text overlap with arXiv:2107.1456

Visual seabed classification using k-means clustering, CIELAB colors and Gabor-filters

In this article, we discuss visual classification using unsupervised learning combined with methods that originate from human vision to divide the Baltic seabed to the soft and hard areas. Seabed classification plays an important role in an understanding the undersea environment. Seabed can be characterized to be as muddy, rocky or sandy. Mine countermeasures (MCM) missions normally are clearance and/or route finding types and in both of these cases successful detection and classification is strongly connected of seabed type. As our unsupervised learning method, we used k-means clustering. When we filtered our gray-scale seabed picture using Gabor filters, we noticed significant improvement after we segmented filtered image with k-means. We will also show results that we achieved using k-means alone and with Lab colors that are designed to approximate human vision

Target detection in clutter for sonar imagery

This thesis is concerned with the analysis of side-looking sonar images, and specif- ically with the identification of the types of seabed that are present in such images, and with the detection of man-made objects in such images. Side-looking sonar images are, broadly speaking, the result of the physical interaction between acous- tic waves and the bottom of the sea. Because of this interaction, the types of seabed appear as textured areas in side-looking sonar images. The texture descrip- tors commonly used in the field of sonar imagery fail at accurately identifying the types of seabed because the types of seabed, hence the textures, are extremely variable. In this thesis, we did not use the traditional texture descriptors to identify the types of seabed. We rather used scattering operators which recently appeared in the field of signal and image processing. We assessed how well the types of seabed are identified through two inference algorithms, one based on affine spaces, and the other based on the concept of similarity by composition. This thesis is also concerned with the detection of man-made objects in side-looking sonar im- ages. An object detector may be described as a method which, when applied to a certain number of sonar images, produces a set of detections. Some of these are true positives, and correspond to real objects. Others are false positives, and do not correspond to real objects. The present object detectors suffer from a high false positive rate in complex environments, that is to say, complex types of seabed. The hypothesis we will follow is that it is possible to reduce the number of false positives through a characterisation of the similarity between the detections and the seabed, the false positives being by nature part of the seabed. We will use scattering operators to represent the detections and the same two inference algorithms to quantify how similar the detections are to the seabed

Sonar image interpretation for sub-sea operations

Mine Counter-Measure (MCM) missions are conducted to neutralise underwater explosives. Automatic Target Recognition (ATR) assists operators by increasing the speed and accuracy of data review. ATR embedded on vehicles enables adaptive missions which increase the speed of data acquisition. This thesis addresses three challenges; the speed of data processing, robustness of ATR to environmental conditions and the large quantities of data required to train an algorithm. The main contribution of this thesis is a novel ATR algorithm. The algorithm uses features derived from the projection of 3D boxes to produce a set of 2D templates. The template responses are independent of grazing angle, range and target orientation. Integer skewed integral images, are derived to accelerate the calculation of the template responses. The algorithm is compared to the Haar cascade algorithm. For a single model of sonar and cylindrical targets the algorithm reduces the Probability of False Alarm (PFA) by 80% at a Probability of Detection (PD) of 85%. The algorithm is trained on target data from another model of sonar. The PD is only 6% lower even though no representative target data was used for training. The second major contribution is an adaptive ATR algorithm that uses local sea-floor characteristics to address the problem of ATR robustness with respect to the local environment. A dual-tree wavelet decomposition of the sea-floor and an Markov Random Field (MRF) based graph-cut algorithm is used to segment the terrain. A Neural Network (NN) is then trained to filter ATR results based on the local sea-floor context. It is shown, for the Haar Cascade algorithm, that the PFA can be reduced by 70% at a PD of 85%. Speed of data processing is addressed using novel pre-processing techniques. The standard three class MRF, for sonar image segmentation, is formulated using graph-cuts. Consequently, a 1.2 million pixel image is segmented in 1.2 seconds. Additionally, local estimation of class models is introduced to remove range dependent segmentation quality. Finally, an A* graph search is developed to remove the surface return, a line of saturated pixels often detected as false alarms by ATR. The A* search identifies the surface return in 199 of 220 images tested with a runtime of 2.1 seconds. The algorithm is robust to the presence of ripples and rocks

Anomaly detection & object classification using multi-spectral LiDAR and sonar

In this thesis, we present the theory of high-dimensional signal approximation of multifrequency signals. We also present both linear and non-linear compressive sensing (CS) algorithms that generate encoded representations of time-correlated single photon counting (TCSPC) light detection and ranging (LiDAR) data, side-scan sonar (SSS) and synthetic aperture sonar (SAS). The main contributions of this thesis are summarised as follows: 1. Research is carried out studying full-waveform (FW) LiDARs, in particular, the TCSPC data, capture, storage and processing. 2. FW-LiDARs are capable of capturing large quantities of photon-counting data in real-time. However, the real-time processing of the raw LiDAR waveforms hasn’t been widely exploited. This thesis answers some of the fundamental questions: • can semantic information be extracted and encoded from raw multi-spectral FW-LiDAR signals? • can these encoded representations then be used for object segmentation and classification? 3. Research is carried out into signal approximation and compressive sensing techniques, its limitations and the application domains. 4. Research is also carried out in 3D point cloud processing, combining geometric features with material spectra (spectral-depth representation), for object segmentation and classification. 5. Extensive experiments have been carried out with publicly available datasets, e.g. the Washington RGB Image and Depth (RGB-D) dataset [108], YaleB face dataset1 [110], real-world multi-frequency aerial laser scans (ALS)2 and an underwater multifrequency (16 wavelengths) TCSPC dataset collected using custom-build targets especially for this thesis. 6. The multi-spectral measurements were made underwater on targets with different shapes and materials. A novel spectral-depth representation is presented with strong discrimination characteristics on target signatures. Several custom-made and realistically scaled exemplars with known and unknown targets have been investigated using a multi-spectral single photon counting LiDAR system. 7. In this work, we also present a new approach to peak modelling and classification for waveform enabled LiDAR systems. Not all existing approaches perform peak modelling and classification simultaneously in real-time. This was tested on both simulated waveform enabled LiDAR data and real ALS data2 . This PhD also led to an industrial secondment at Carbomap, Edinburgh, where some of the waveform modelling algorithms were implemented in C++ and CUDA for Nvidia TX1 boards for real-time performance. 1http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ 2This dataset was captured in collaboration with Carbomap Ltd. Edinburgh, UK. The data was collected during one of the trials in Austria using commercial-off-the-shelf (COTS) sensors

Development of an unsupervised remote sensing methodology of detect surface leakage from terrestrial CO2 storage sites

Imperial Users onl

Computational imaging and automated identification for aqueous environments

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2011Sampling the vast volumes of the ocean requires tools capable of observing from a distance while retaining detail necessary for biology and ecology, ideal for optical methods. Algorithms that work with existing SeaBED AUV imagery are developed, including habitat classi fication with bag-of-words models and multi-stage boosting for rock sh detection. Methods for extracting images of sh from videos of longline operations are demonstrated. A prototype digital holographic imaging device is designed and tested for quantitative in situ microscale imaging. Theory to support the device is developed, including particle noise and the effects of motion. A Wigner-domain model provides optimal settings and optical limits for spherical and planar holographic references. Algorithms to extract the information from real-world digital holograms are created. Focus metrics are discussed, including a novel focus detector using local Zernike moments. Two methods for estimating lateral positions of objects in holograms without reconstruction are presented by extending a summation kernel to spherical references and using a local frequency signature from a Riesz transform. A new metric for quickly estimating object depths without reconstruction is proposed and tested. An example application, quantifying oil droplet size distributions in an underwater plume, demonstrates the efficacy of the prototype and algorithms.Funding was provided by NOAA Grant #5710002014, NOAA NMFS Grant #NA17RJ1223, NSF Grant #OCE-0925284, and NOAA Grant #NA10OAR417008

Object-based mapping of temperate marine habitats from multi-resolution remote sensing data

PhD ThesisHabitat maps are needed to inform marine spatial planning but current methods of field survey and data interpretation are time-consuming and subjective. Object-based image analysis (OBIA) and remote sensing could deliver objective, cost-effective solutions informed by ecological knowledge. OBIA enables development of automated workflows to segment imagery, creating ecologically meaningful objects which are then classified based on spectral or geometric properties, relationships to other objects and contextual data. Successfully applied to terrestrial and tropical marine habitats for over a decade, turbidity and lack of suitable remotely sensed data had limited OBIA’s use in temperate seas to date. This thesis evaluates the potential of OBIA and remote sensing to inform designation, management and monitoring of temperate Marine Protected Areas (MPAs) through four studies conducted in English North Sea MPAs. An initial study developed OBIA workflows to produce circalittoral habitat maps from acoustic data using sequential threshold-based and nearest neighbour classifications. These methods produced accurate substratum maps over large areas but could not reliably predict distribution of species communities from purely physical data under largely homogeneous environmental conditions. OBIA methods were then tested in an intertidal MPA with fine-scale habitat heterogeneity using high resolution imagery collected by unmanned aerial vehicle. Topographic models were created from the imagery using photogrammetry. Validation of these models through comparison with ground truth measurements showed high vertical accuracy and the ability to detect decimetre-scale features. The topographic and spectral layers were interpreted simultaneously using OBIA, producing habitat maps at two thematic scales. Classifier comparison showed that Random Forests Abstract ii outperformed the nearest neighbour approach, while a knowledge-based rule set produced accurate results but requires further research to improve reproducibility. The final study applied OBIA methods to aerial and LiDAR time-series, demonstrating that despite considerable variability in the data, pre- and post-classification change detection methods had sufficient accuracy to monitor deviation from a background level of natural environmental fluctuation. This thesis demonstrates the potential of OBIA and remote sensing for large-scale rapid assessment, detailed surveillance and change detection, providing insight to inform choice of classifier, sampling protocol and thematic scale which should aid wider adoption of these methods in temperate MPAs.Natural Environment Research Council and Natural Englan

Computational imaging and automated identification for aqueous environments

Thesis (Ph. D.)--Joint Program in Oceanography/Applied Ocean Science and Engineering (Massachusetts Institute of Technology, Dept. of Mechanical Engineering; and the Woods Hole Oceanographic Institution), 2011."June 2011." Cataloged from PDF version of thesis.Includes bibliographical references (p. 253-293).Sampling the vast volumes of the ocean requires tools capable of observing from a distance while retaining detail necessary for biology and ecology, ideal for optical methods. Algorithms that work with existing SeaBED AUV imagery are developed, including habitat classification with bag-of-words models and multi-stage boosting for rock sh detection. Methods for extracting images of sh from videos of long-line operations are demonstrated. A prototype digital holographic imaging device is designed and tested for quantitative in situ microscale imaging. Theory to support the device is developed, including particle noise and the effects of motion. A Wigner-domain model provides optimal settings and optical limits for spherical and planar holographic references. Algorithms to extract the information from real-world digital holograms are created. Focus metrics are discussed, including a novel focus detector using local Zernike moments. Two methods for estimating lateral positions of objects in holograms without reconstruction are presented by extending a summation kernel to spherical references and using a local frequency signature from a Riesz transform. A new metric for quickly estimating object depths without reconstruction is proposed and tested. An example application, quantifying oil droplet size distributions in an underwater plume, demonstrates the efficacy of the prototype and algorithms.by Nicholas C. Loomis.Ph.D