Quantitative Comparison of Benthic Habitat Maps Derived From Multibeam Echosounder Backscatter Data

In the last decade, following the growing concern for the conservation of marine ecosystems, a wide range of approaches has been developed to achieve the identification, classification and mapping of seabed types and of benthic habitats. These approaches, commonly grouped under the denominations of Benthic Habitat Mapping or Acoustic Seabed Classification, exploit the latest scientific and engineering advancements for the exploration of the bottom of the ocean, particularly in underwater acoustics. Among all acoustic seabed-mapping systems available for this purpose, a growing interest has recently developed for Multibeam Echosounders (MBES). This interest is mainly the result of the multiplicity of these systems’ outputs (that is, bathymetry, backscatter mosaic, angular response and water-column data), which allows for multiple approaches to seabed or habitat classification and mapping. While this diversity of mapping approaches and this multiplicity of MBES data products contribute to an increasing quality of the charting of the marine environment, they also unfortunately delay the future standardization of mapping methods, which is required for their effective integration in marine environment management strategies. As a preliminary step towards such standardization, there is a need for generalized efforts of comparison of systems, data products, and mapping approaches, in order to assess the most effective ones given mapping objectives and environment conditions. The main goal of this thesis is to contribute to this effort through the development and implementation of tools and methods for the comparison of categorical seabed or habitat maps, with a specific focus on maps obtained from up-to-date methodologies of classification of MBES backscatter data. This goal is attained through the achievement of specific objectives treated sequentially. First, the need for comparison is justified through a review of the diversity characterizing the fields of Benthic Habitat Mapping and Acoustic Seabed Classification, and of their use of MBES data products. Then, a case study is presented that compare the data products from a Kongsberg EM3000 MBES to the output map of an Acoustic Ground Discrimination Software based on data from a Single-beam Echosounder and to a Sidescan Sonar mosaic, in order to illustrate how map comparison measures could contribute to the comparison of these systems. Next, a number of measures for map-to-map comparison, inspired from the literature in land remote sensing, are presented, along with methodologies for their implementation in comparison of maps described with different legends. The benefit of these measures and methodologies is demonstrated through their application to maps obtained from the acoustic datasets presented previously. Finally, a more typical implementation of these measures is presented as a case study in which the development of two up-to-date classification methodologies of MBES backscatter data is complemented by the quantitative comparison of their output maps. In the process of developing and illustrating the use of methods for the assessment of map-to-map similarity, this thesis also presents methodologies for the processing and classification of backscatter data from MBES. In particular, the potential of the combined use of the spatial and angular information of these data for seabed classification is explored through the development of an original segmentation methodology that sequentially divides and aggregates segments defined from a MBES backscatter mosaic on the basis of their angular response content

Machine learning methods for discriminating natural targets in seabed imagery

The research in this thesis concerns feature-based machine learning processes and methods for discriminating qualitative natural targets in seabed imagery. The applications considered, typically involve time-consuming manual processing stages in an industrial setting. An aim of the research is to facilitate a means of assisting human analysts by expediting the tedious interpretative tasks, using machine methods. Some novel approaches are devised and investigated for solving the application problems. These investigations are compartmentalised in four coherent case studies linked by common underlying technical themes and methods. The first study addresses pockmark discrimination in a digital bathymetry model. Manual identification and mapping of even a relatively small number of these landform objects is an expensive process. A novel, supervised machine learning approach to automating the task is presented. The process maps the boundaries of ≈ 2000 pockmarks in seconds - a task that would take days for a human analyst to complete. The second case study investigates different feature creation methods for automatically discriminating sidescan sonar image textures characteristic of Sabellaria spinulosa colonisation. Results from a comparison of several textural feature creation methods on sonar waterfall imagery show that Gabor filter banks yield some of the best results. A further empirical investigation into the filter bank features created on sonar mosaic imagery leads to the identification of a useful configuration and filter parameter ranges for discriminating the target textures in the imagery. Feature saliency estimation is a vital stage in the machine process. Case study three concerns distance measures for the evaluation and ranking of features on sonar imagery. Two novel consensus methods for creating a more robust ranking are proposed. Experimental results show that the consensus methods can improve robustness over a range of feature parameterisations and various seabed texture classification tasks. The final case study is more qualitative in nature and brings together a number of ideas, applied to the classification of target regions in real-world sonar mosaic imagery. A number of technical challenges arose and these were surmounted by devising a novel, hybrid unsupervised method. This fully automated machine approach was compared with a supervised approach in an application to the problem of image-based sediment type discrimination. The hybrid unsupervised method produces a plausible class map in a few minutes of processing time. It is concluded that the versatile, novel process should be generalisable to the discrimination of other subjective natural targets in real-world seabed imagery, such as Sabellaria textures and pockmarks (with appropriate features and feature tuning.) Further, the full automation of pockmark and Sabellaria discrimination is feasible within this framework

Ocean remote sensing techniques and applications: a review (Part II)

As discussed in the first part of this review paper, Remote Sensing (RS) systems are great tools to study various oceanographic parameters. Part I of this study described different passive and active RS systems and six applications of RS in ocean studies, including Ocean Surface Wind (OSW), Ocean Surface Current (OSC), Ocean Wave Height (OWH), Sea Level (SL), Ocean Tide (OT), and Ship Detection (SD). In Part II, the remaining nine important applications of RS systems for ocean environments, including Iceberg, Sea Ice (SI), Sea Surface temperature (SST), Ocean Surface Salinity (OSS), Ocean Color (OC), Ocean Chlorophyll (OCh), Ocean Oil Spill (OOS), Underwater Ocean, and Fishery are comprehensively reviewed and discussed. For each application, the applicable RS systems, their advantages and disadvantages, various RS and Machine Learning (ML) techniques, and several case studies are discussed.Peer ReviewedPostprint (published version

Automatic target recognition in sonar imagery using a cascade of boosted classifiers

This thesis is concerned with the problem of automating the interpretation of data representing the underwater environment retrieved from sensors. This is an important task which potentially allows underwater robots to become completely autonomous, keeping humans out of harm’s way and reducing the operational time and cost of many underwater applications. Typical applications include unexploded ordnance clearance, ship/plane wreck hunting (e.g. Malaysia Airlines flight MH370), and oilfield inspection (e.g. Deepwater Horizon disaster). Two attributes of the processing are crucial if automated interpretation is to be successful. First, computational efficiency is required to allow real-time analysis to be performed on-board robots with limited resources. Second, detection accuracy comparable to human experts is required in order to replace them. Approaches in the open literature do not appear capable of achieving these requirements and this therefore has become the objective of this thesis. This thesis proposes a novel approach capable of recognizing targets in sonar data extremely rapidly with a low number of false alarms. The approach was originally developed for face detection in video, and it is applied to sonar data here for the first time. Aside from the application, the main contribution of this thesis, therefore, is in the way this approach is extended to reduce its training time and improve its detection accuracy. Results obtained on large sets of real sonar data on a variety of challenging terrains are presented to show the discriminative power of the proposed approach. In real field trials, the proposed approach was capable of processing sonar data real-time on-board underwater robots. In direct comparison with human experts, the proposed approach offers 40% reduction in the number of false alarms

Acoustic data optimisation for seabed mapping with visual and computational data mining

Oceans cover 70% of Earth’s surface but little is known about their waters. While the echosounders, often used for exploration of our oceans, have developed at a tremendous rate since the WWII, the methods used to analyse and interpret the data still remain the same. These methods are inefficient, time consuming, and often costly in dealing with the large data that modern echosounders produce. This PhD project will examine the complexity of the de facto seabed mapping technique by exploring and analysing acoustic data with a combination of data mining and visual analytic methods. First we test the redundancy issues in multibeam echosounder (MBES) data by using the component plane visualisation of a Self Organising Map (SOM). A total of 16 visual groups were identified among the 132 statistical data descriptors. The optimised MBES dataset had 35 attributes from 16 visual groups and represented a 73% reduction in data dimensionality. A combined Principal Component Analysis (PCA) + k-means was used to cluster both the datasets. The cluster results were visually compared as well as internally validated using four different internal validation methods. Next we tested two novel approaches in singlebeam echosounder (SBES) data processing and clustering – using visual exploration for outlier detection and direct clustering of time series echo returns. Visual exploration identified further outliers the automatic procedure was not able to find. The SBES data were then clustered directly. The internal validation indices suggested the optimal number of clusters to be three. This is consistent with the assumption that the SBES time series represented the subsurface classes of the seabed. Next the SBES data were joined with the corresponding MBES data based on identification of the closest locations between MBES and SBES. Two algorithms, PCA + k-means and fuzzy c-means were tested and results visualised. From visual comparison, the cluster boundary appeared to have better definitions when compared to the clustered MBES data only. The results seem to indicate that adding SBES did in fact improve the boundary definitions. Next the cluster results from the analysis chapters were validated against ground truth data using a confusion matrix and kappa coefficients. For MBES, the classes derived from optimised data yielded better accuracy compared to that of the original data. For SBES, direct clustering was able to provide a relatively reliable overview of the underlying classes in survey area. The combined MBES + SBES data provided by far the best accuracy for mapping with almost a 10% increase in overall accuracy compared to that of the original MBES data. The results proved to be promising in optimising the acoustic data and improving the quality of seabed mapping. Furthermore, these approaches have the potential of significant time and cost saving in the seabed mapping process. Finally some future directions are recommended for the findings of this research project with the consideration that this could contribute to further development of seabed mapping problems at mapping agencies worldwide

Machine learning in marine ecology: an overview of techniques and applications

Machine learning covers a large set of algorithms that can be trained to identify patterns in data. Thanks to the increase in the amount of data and computing power available, it has become pervasive across scientific disciplines. We first highlight why machine learning is needed in marine ecology. Then we provide a quick primer on machine learning techniques and vocabulary. We built a database of ∼1000 publications that implement such techniques to analyse marine ecology data. For various data types (images, optical spectra, acoustics, omics, geolocations, biogeochemical profiles, and satellite imagery), we present a historical perspective on applications that proved influential, can serve as templates for new work, or represent the diversity of approaches. Then, we illustrate how machine learning can be used to better understand ecological systems, by combining various sources of marine data. Through this coverage of the literature, we demonstrate an increase in the proportion of marine ecology studies that use machine learning, the pervasiveness of images as a data source, the dominance of machine learning for classification-type problems, and a shift towards deep learning for all data types. This overview is meant to guide researchers who wish to apply machine learning methods to their marine datasets.Machine learning in marine ecology: an overview of techniques and applicationspublishedVersio

Coastal Hurricane Damage Assessment via Wavelet Transform of Remotely Sensed Imagery

This dissertation uses post storm imagery processed using wavelet transforms to investigate the capability of wavelet transform-based methods to classify post storm damage of residential areas. Five level Haar, Meyer, Symlets, and Coiflets wavelet transform decompositions of the post storm imagery are inputs to damage classification models of post hurricane and tornado damage. Hurricanes Ike, Rita, Katrina, and Ivan are examined as are the 2011 Joplin and Tuscaloosa tornadoes. Wavelet transform-based classification methods yielded varying classification accuracies for the four hurricanes examined, ranging from 67 percent to 89 percent classification accuracy for classification models informed by samples from the storms classified. Classification accuracies fall when the samples being classified are from a hurricane not informing the classification model, from 17 percent for Rita classified with an Ike-based model, 41 percent for Rita classified with an Ike-Katrina-based model, to 69 percent for Rita classified with an Ike-Katrina-Ivan-based model. The variability within and poor classification accuracy of these models can be attributed to the large variations in the four hurricane events studied and the significant differences in impacted land cover for each of these storms. Classification accuracies improved when these variations were limited via examination of residential areas impacted by 2011 Joplin and Tuscaloosa tornadoes. Damage classification models required as few as nineteen to as many as fifty nine wavelet coefficients to explain the variability in the hurricane storm data samples, and included all four wavelet functions studied. A similar analysis of the tornado damaged areas resulted in a damage classification model with only six wavelet coefficients, four Meyer-based, one Symlets-based and one Haar-based. Classification accuracies ranged from 96 percent for samples included in the model formation to 85 percent for samples not included in the model formation. The damage classification accuracies found for tornado storms suggests this model is suitable for operational implementation. The damage classification accuracies found for the hurricane storms suggests further investigation into methods that will reduce the variability attributable to land cover and storm variability

Contributions à la sonification d’image et à la classification de sons

L’objectif de cette thèse est d’étudier d’une part le problème de sonification d’image et de le solutionner à travers de nouveaux modèles de correspondance entre domaines visuel et sonore. D’autre part d’étudier le problème de la classification de son et de le résoudre avec des méthodes ayant fait leurs preuves dans le domaine de la reconnaissance d’image. La sonification d’image est la traduction de données d’image (forme, couleur, texture, objet) en sons. Il est utilisé dans les domaines de l’assistance visuelle et de l’accessibilité des images pour les personnes malvoyantes. En raison de sa complexité, un système de sonification d’image qui traduit correctement les données d’image en son de manière intuitive n’est pas facile à concevoir. Notre première contribution est de proposer un nouveau système de sonification d’image de bas-niveau qui utilise une approche hiérarchique basée sur les caractéristiques visuelles. Il traduit, à l’aide de notes musicales, la plupart des propriétés d’une image (couleur, gradient, contour, texture, région) vers le domaine audio, de manière très prévisible et donc est facilement ensuite décodable par l’être humain. Notre deuxième contribution est une application Android de sonification de haut niveau qui est complémentaire à notre première contribution car elle implémente la traduction des objets et du contenu sémantique de l’image. Il propose également une base de données pour la sonification d’image. Finalement dans le domaine de l’audio, notre dernière contribution généralise le motif binaire local (LBP) à 1D et le combine avec des descripteurs audio pour faire de la classification de sons environnementaux. La méthode proposée surpasse les résultats des méthodes qui utilisent des algorithmes d’apprentissage automatique classiques et est plus rapide que toutes les méthodes de réseau neuronal convolutif. Il représente un meilleur choix lorsqu’il y a une rareté des données ou une puissance de calcul minimale.The objective of this thesis is to study on the one hand the problem of image sonification and to solve it through new models of mapping between visual and sound domains. On the other hand, to study the problem of sound classification and to solve it with methods which have proven track record in the field of image recognition. Image sonification is the translation of image data (shape, color, texture, objects) into sounds. It is used in vision assistance and image accessibility domains for visual impaired people. Due to its complexity, an image sonification system that properly conveys the image data to sound in an intuitive way is not easy to design. Our first contribution is to propose a new low-level image sonification system which uses an hierarchical visual feature-based approach to translate, usingmusical notes, most of the properties of an image (color, gradient, edge, texture, region) to the audio domain, in a very predictable way in which is then easily decodable by the human being. Our second contribution is a high-level sonification Android application which is complementary to our first contribution because it implements the translation to the audio domain of the objects and the semantic content of an image. It also proposes a dataset for an image sonification. Finally, in the audio domain, our third contribution generalizes the Local Binary Pattern (LBP) to 1D and combines it with audio features for an environmental sound classification task. The proposed method outperforms the results of methods that uses handcrafted features with classical machine learning algorithms and is faster than any convolutional neural network methods. It represents a better choice when there is data scarcity or minimal computing power