Machine learning methods for discriminating natural targets in seabed imagery

The research in this thesis concerns feature-based machine learning processes and methods for discriminating qualitative natural targets in seabed imagery. The applications considered, typically involve time-consuming manual processing stages in an industrial setting. An aim of the research is to facilitate a means of assisting human analysts by expediting the tedious interpretative tasks, using machine methods. Some novel approaches are devised and investigated for solving the application problems. These investigations are compartmentalised in four coherent case studies linked by common underlying technical themes and methods. The first study addresses pockmark discrimination in a digital bathymetry model. Manual identification and mapping of even a relatively small number of these landform objects is an expensive process. A novel, supervised machine learning approach to automating the task is presented. The process maps the boundaries of ≈ 2000 pockmarks in seconds - a task that would take days for a human analyst to complete. The second case study investigates different feature creation methods for automatically discriminating sidescan sonar image textures characteristic of Sabellaria spinulosa colonisation. Results from a comparison of several textural feature creation methods on sonar waterfall imagery show that Gabor filter banks yield some of the best results. A further empirical investigation into the filter bank features created on sonar mosaic imagery leads to the identification of a useful configuration and filter parameter ranges for discriminating the target textures in the imagery. Feature saliency estimation is a vital stage in the machine process. Case study three concerns distance measures for the evaluation and ranking of features on sonar imagery. Two novel consensus methods for creating a more robust ranking are proposed. Experimental results show that the consensus methods can improve robustness over a range of feature parameterisations and various seabed texture classification tasks. The final case study is more qualitative in nature and brings together a number of ideas, applied to the classification of target regions in real-world sonar mosaic imagery. A number of technical challenges arose and these were surmounted by devising a novel, hybrid unsupervised method. This fully automated machine approach was compared with a supervised approach in an application to the problem of image-based sediment type discrimination. The hybrid unsupervised method produces a plausible class map in a few minutes of processing time. It is concluded that the versatile, novel process should be generalisable to the discrimination of other subjective natural targets in real-world seabed imagery, such as Sabellaria textures and pockmarks (with appropriate features and feature tuning.) Further, the full automation of pockmark and Sabellaria discrimination is feasible within this framework

Seafloor Segmentation Based on Bathymetric Measurements from Multibeam Echosounders Data

Bathymetric data depicts the geomorphology of the seabottom and allows characterization of spatial distributions of apparent benthic habitats. The variability of seafloor topography can be defined as a texture. This prompts for the application of well developed image processing techniques for automatic delineation of regions with clucially different physiographic characteristics. In the present paper histograms of biologically motivated invariant image attributes are used for characterization of local geomorphological feahires. This technique can be naturally applied in a range of spatial scales. Local feature vectors are then submitted to a procedure which divides the set into a number of clusters each representing a distinct type of the seafloor. Prior knowledge about benthic habitat locations allows the use of supervised classification, by training a Suppolt Vector Machine on a chosen data set, and then applying the developed model to a full set. The classification method is shown to perform well on the multibeam echosounder (MBES) data from Piscataqua River, New Hampshire, USA

Assessing the repeatability of automated seafloor classification algorithms, with application in marine protected area monitoring

The number and areal extent of marine protected areas worldwide is rapidly increasing as a result of numerous national targets that aim to see up to 30% of their waters protected by 2030. Automated seabed classification algorithms are arising as faster and objective methods to generate benthic habitat maps to monitor these areas. However, no study has yet systematically compared their repeatability. Here we aim to address that problem by comparing the repeatability of maps derived from acoustic datasets collected on consecutive days using three automated seafloor classification algorithms: (1) Random Forest (RF), (2) K–Nearest Neighbour (KNN) and (3) K means (KMEANS). The most robust and repeatable approach is then used to evaluate the change in seafloor habitats between 2012 and 2015 within the Greater Haig Fras Marine Conservation Zone, Celtic Sea, UK. Our results demonstrate that only RF and KNN provide statistically repeatable maps, with 60.3% and 47.2% agreement between consecutive days. Additionally, this study suggests that in low-relief areas, bathymetric derivatives are non-essential input parameters, while backscatter textural features, in particular Grey Level Co-occurrence Matrices, are substantially more effective in the detection of different habitats. Habitat persistence in the test area between 2012 and 2015 was 48.8%, with swapping of habitats driving the changes in 38.2% of the area. Overall, this study highlights the importance of investigating the repeatability of automated seafloor classification methods before they can be fully used in the monitoring of benthic habitat

Overview: Computer vision and machine learning for microstructural characterization and analysis

The characterization and analysis of microstructure is the foundation of microstructural science, connecting the materials structure to its composition, process history, and properties. Microstructural quantification traditionally involves a human deciding a priori what to measure and then devising a purpose-built method for doing so. However, recent advances in data science, including computer vision (CV) and machine learning (ML) offer new approaches to extracting information from microstructural images. This overview surveys CV approaches to numerically encode the visual information contained in a microstructural image, which then provides input to supervised or unsupervised ML algorithms that find associations and trends in the high-dimensional image representation. CV/ML systems for microstructural characterization and analysis span the taxonomy of image analysis tasks, including image classification, semantic segmentation, object detection, and instance segmentation. These tools enable new approaches to microstructural analysis, including the development of new, rich visual metrics and the discovery of processing-microstructure-property relationships.Comment: submitted to Materials and Metallurgical Transactions

Fuzzy Based Texton Binary Shape Matrix (FTBSM) for Texture Classification

Texton is a extensively applied approach for texture analysis. This technique shows a strong dependence on certain number of parameters. Unfortunately, each variation of values of any parameter may affect the texture characterization performance. Moreover, micro structure texton is unable to extract texture features which also have a negative effect on the classification task. This paper, deals with a new descriptor which avoids the drawbacks mentioned above. To address the above, the present paper derives a new descriptor called Fuzzy Based Texton Binary Shape Matrix (FTBSM) for clear variation of any feature/parameter. The proposed FTBSM are defined based on similarity of neighboring edges on a 3D7;3 neighborhood. With micro-structures serving as a bridge for extracting shape features and it effectively integrates color, texture and shape component information as a whole for texture classification. The proposed FTBSM algorithm exhibits low dimensionality. The proposed FTBSM method is tested on Vistex and Akarmarble texture datasets of natural images. The results demonstrate that it is much more efficient and effective than representative feature descriptors, such as logical operators and GLCM and LBP, for texture classification

Sea ice detection using concurrent multispectral and synthetic aperture radar imagery

Synthetic Aperture Radar (SAR) imagery is the primary data type used for sea ice mapping due to its spatio-temporal coverage and the ability to detect sea ice independent of cloud and lighting conditions. Automatic sea ice detection using SAR imagery remains problematic due to the presence of ambiguous signal and noise within the image. Conversely, ice and water are easily distinguishable using multispectral imagery (MSI), but in the polar regions the ocean's surface is often occluded by cloud or the sun may not appear above the horizon for many months. To address some of these limitations, this paper proposes a new tool trained using concurrent multispectral Visible and SAR imagery for sea Ice Detection (ViSual\_IceD). ViSual\_IceD is a convolution neural network (CNN) that builds on the classic U-Net architecture by containing two parallel encoder stages, enabling the fusion and concatenation of MSI and SAR imagery containing different spatial resolutions. The performance of ViSual\_IceD is compared with U-Net models trained using concatenated MSI and SAR imagery as well as models trained exclusively on MSI or SAR imagery. ViSual\_IceD outperforms the other networks, with a F1 score 1.60\% points higher than the next best network, and results indicate that ViSual\_IceD is selective in the image type it uses during image segmentation. Outputs from ViSual\_IceD are compared to sea ice concentration products derived from the AMSR2 Passive Microwave (PMW) sensor. Results highlight how ViSual\_IceD is a useful tool to use in conjunction with PMW data, particularly in coastal regions. As the spatial-temporal coverage of MSI and SAR imagery continues to increase, ViSual\_IceD provides a new opportunity for robust, accurate sea ice coverage detection in polar regions.Comment: 34 pages, 10 figures, 2 table

Evaluation of seabed mapping methods for fine-scale classification of extremely shallow benthic habitats – application to the Venice Lagoon, Italy

Recent technological developments of multibeam echosounder systems (MBES) allow mapping of benthic habitats with unprecedented detail. MBES can now be employed in extremely shallow waters, challenging data acquisition (as these instruments were often designed for deeper waters) and data interpretation (honed on datasets with resolution sometimes orders of magnitude lower). With extremely high-resolution bathymetry and colocated backscatter data, it is now possible to map the spatial distribution of fine scale benthic habitats, even identifying the acoustic signatures of single sponges. In this context, it isnecessary to understand which of the commonly used segmentation methods is best suited to account for such level of detail. At the same time, new sampling protocols for precisely georeferenced ground truth data need to be developed to validate the benthic environmental classification. This study focuses on a dataset collected in a shallow (2–10 m deep) tidal channel of the Lagoon of Venice, Italy. Using 0.05-m and 0.2-m raster grids, we compared a range of classifications, both pixel- based and object-based approaches, including manual, Maximum Likelihood Classifier, Jenks Optimization clustering, textural analysis and ObjectBased Image Analysis. Through a comprehensive and accurately geo-referenced ground truth dataset, we were able to identify five different classes of the substrate composition, including sponges, mixed submerged aquatic vegetation, mixed detritic bottom (fine and coarse) and unconsolidated bare sediment. We computed estimates of accuracy (namely Overall, User and Producer Accuracies) by cross tabulating predicted and reference instances. Overall, pixel based segmentations produced the highest accuracies and that the accuracy assessment is strongly dependent on the choice of classes for the segmentation. Tidal channels in the Venice Lagoon are extremely important in terms of habitats and sediment distribution, particularly within the context of the new tidal barrier being built. However, they had remained largely unexplored until now, because of the surveying challenges. The application of this remote sensing approach, combined with targeted sampling, opens a new perspective in the monitoring of benthic habitats in view of a knowledge-based management of natural resources in shallow coastal areas

Habitat mapping of the Maltese continental shelf using acoustic textures and bathymetric analyses

The uneven mapping of the Maltese continental shelf precludes a full assessment of its marine habitats, important for their monitoring and conservation in line with the EU Marine Strategy Framework Directive and local initiatives. From 2009 to 2012, high-resolution multibeam echosounder (MBES) surveys offshore the NW and E coasts of the Maltese archipelago were carried out, covering a total area of 1,408.3 km2 with a maximum resolution of 1 m, at depths from 1.5 to 263 m. The types of benthic habitats occurring on the continental shelf often showed subtle acoustic variations. This article aims at 1) integrating analyses of the bathymetry and acoustic textures with ground-truthing (grab samples) in key areas; 2) validating this combined approach by rewriting an existing benthic habitat map of the eastern continental shelf of Malta; 3) exploiting this ground-truthed classification to calibrate an unsupervised classification of a dataset acquired with a different sonar. The main results obtained from these analyses are i) a sediment map of the continental shelf of NW Malta and east of the Maltese archipelago – classifying in detail bedrock, rocky blocks, coarse sand and gravel, fine to medium sand and maërl, sand and gravel – that supports the geomorphological interpretation of the seabed features; ii) an automatic classification of the seafloor morphology, highlighting a very gentle sloping seabed crossed by the shelf break and by palaeo-river valleys; iii) the first full benthic habitat map of the continental shelf offshore E and NW coast of Malta obtained with a semi-automatic classification. In this work, we highlight and explain the main differences in seafloor sediment coverage, its morphology and the relative occurrences of benthic habitats between the NW and E sides of the Maltese archipelago

Experimental and Data-driven Workflows for Microstructure-based Damage Prediction

Materialermüdung ist die häufigste Ursache für mechanisches Versagen. Die Degradationsmechanismen, welche die Lebensdauer von Bauteilen bei vergleichsweise ausgeprägten zyklischen Belastungen bestimmen, sind gut bekannt. Bei Belastungen im makroskopisch elastischen Bereich hingegen, der (sehr) hochzyklischen Ermüdung, bestimmen die innere Struktur eines Werkstoffs und die Wechselwirkung kristallografischer Defekte die Lebensdauer. Unter diesen Umständen sind die inneren Degradationsphänomene auf der mikroskopischen Skala weitgehend reversibel und führen nicht zur Bildung kritischer Schädigungen, die kontinuierlich wachsen können. Allerdings sind einige Kornensembles in polykristallinen Metallen, je nach den lokalen mikrostrukturellen Gegebenheiten, anfällig für Schädigungsinitiierung, Rissbildung und -wachstum und wirken daher als Schwachstellen. Daher weisen Bauteile, die solchen Belastungen ausgesetzt sind, oft eine ausgeprägte Lebensdauerstreuung auf. Die Tatsache, dass ein umfassendes mechanistisches Verständnis für diese Degradationsprozesse in verschiedenen Werkstoffen nicht vorliegt, hat zur Folge, dass die derzeitigen Modellierungsbemühungen die mittlere Lebensdauer und ihre Varianz in der Regel nur mit unbefriedigender Genauigkeit vorhersagen. Dies wiederum erschwert die Bauteilauslegung und macht die Nutzung von Sicherheitsfaktoren während des Dimensionierungsprozesses erforderlich. Abhilfe kann geschaffen werden, indem umfangreiche Daten zu Einflussfaktoren und deren Wirkung auf die Bildung initialer Ermüdungsschädigungen erhoben werden. Die Datenknappheit wirkt sich nach wie vor negativ auf Datenwissenschaftler und Modellierungsexperten aus, die versuchen, trotz geringer Stichprobengröße und unvollständigen Merkmalsräumen, mikrostrukturelle Abhängigkeiten abzuleiten, datengetriebene Vorhersagemodelle zu trainieren oder physikalische, regelbasierte Modelle zu parametrisieren. Die Tatsache, dass nur wenige kritische Schädigungen bezogen auf das gesamte Probenvolumen auftreten und die hochzyklische Ermüdung eine Vielzahl unterschiedlicher Abhängigkeiten aufweist, impliziert einige Anforderungen an die Datenerfassung und -verarbeitung. Am wichtigsten ist, dass die Messtechniken so empfindlich sind, dass nuancierte Schwankungen im Probenzustand erfasst werden können, dass die gesamte Routine effizient ist und dass die korrelative Mikroskopie räumliche Informationen aus verschiedenen Messungen miteinander verbindet. Das Hauptziel dieser Arbeit besteht darin, einen Workflow zu etablieren, der den Datenmangel behebt, so dass die zukünftige virtuelle Auslegung von Komponenten effizienter, zuverlässiger und nachhaltiger gestaltet werden kann. Zu diesem Zweck wird in dieser Arbeit ein kombinierter experimenteller und datenverarbeitender Workflow vorgeschlagen, um multimodale Datensätze zu Ermüdungsschädigungen zu erzeugen. Der Schwerpunkt liegt dabei auf dem Auftreten von lokalen Gleitbändern, der Rissinitiierung und dem Wachstum mikrostrukturell kurzer Risse. Der Workflow vereint die Ermüdungsprüfung von mesoskaligen Proben, um die Empfindlichkeit der Schädigungsdetektion zu erhöhen, die ergänzende Charakterisierung, die multimodale Registrierung und Datenfusion der heterogenen Daten, sowie die bildverarbeitungsbasierte Schädigungslokalisierung und -bewertung. Mesoskalige Biegeresonanzprüfung ermöglicht das Erreichen des hochzyklischen Ermüdungszustands in vergleichsweise kurzen Zeitspannen bei gleichzeitig verbessertem Auflösungsvermögen der Schädigungsentwicklung. Je nach Komplexität der einzelnen Bildverarbeitungsaufgaben und Datenverfügbarkeit werden entweder regelbasierte Bildverarbeitungsverfahren oder Repräsentationslernen gezielt eingesetzt. So sorgt beispielsweise die semantische Segmentierung von Schädigungsstellen dafür, dass wichtige Ermüdungsmerkmale aus mikroskopischen Abbildungen extrahiert werden können. Entlang des Workflows wird auf einen hohen Automatisierungsgrad Wert gelegt. Wann immer möglich, wurde die Generalisierbarkeit einzelner Workflow-Elemente untersucht. Dieser Workflow wird auf einen ferritischen Stahl (EN 1.4003) angewendet. Der resultierende Datensatz verknüpft unter anderem große verzerrungskorrigierte Mikrostrukturdaten mit der Schädigungslokalisierung und deren zyklischer Entwicklung. Im Zuge der Arbeit wird der Datensatz wird im Hinblick auf seinen Informationsgehalt untersucht, indem detaillierte, analytische Studien zur einzelnen Schädigungsbildung durchgeführt werden. Auf diese Weise konnten unter anderem neuartige, quantitative Erkenntnisse über mikrostrukturinduzierte plastische Verformungs- und Rissstopmechanismen gewonnen werden. Darüber hinaus werden aus dem Datensatz abgeleitete kornweise Merkmalsvektoren und binäre Schädigungskategorien verwendet, um einen Random-Forest-Klassifikator zu trainieren und dessen Vorhersagegüte zu bewerten. Der vorgeschlagene Workflow hat das Potenzial, die Grundlage für künftiges Data Mining und datengetriebene Modellierung mikrostrukturempfindlicher Ermüdung zu legen. Er erlaubt die effiziente Erhebung statistisch repräsentativer Datensätze mit gleichzeitig hohem Informationsgehalt und kann auf eine Vielzahl von Werkstoffen ausgeweitet werden

Quantitative Comparison of Benthic Habitat Maps Derived From Multibeam Echosounder Backscatter Data

In the last decade, following the growing concern for the conservation of marine ecosystems, a wide range of approaches has been developed to achieve the identification, classification and mapping of seabed types and of benthic habitats. These approaches, commonly grouped under the denominations of Benthic Habitat Mapping or Acoustic Seabed Classification, exploit the latest scientific and engineering advancements for the exploration of the bottom of the ocean, particularly in underwater acoustics. Among all acoustic seabed-mapping systems available for this purpose, a growing interest has recently developed for Multibeam Echosounders (MBES). This interest is mainly the result of the multiplicity of these systems’ outputs (that is, bathymetry, backscatter mosaic, angular response and water-column data), which allows for multiple approaches to seabed or habitat classification and mapping. While this diversity of mapping approaches and this multiplicity of MBES data products contribute to an increasing quality of the charting of the marine environment, they also unfortunately delay the future standardization of mapping methods, which is required for their effective integration in marine environment management strategies. As a preliminary step towards such standardization, there is a need for generalized efforts of comparison of systems, data products, and mapping approaches, in order to assess the most effective ones given mapping objectives and environment conditions. The main goal of this thesis is to contribute to this effort through the development and implementation of tools and methods for the comparison of categorical seabed or habitat maps, with a specific focus on maps obtained from up-to-date methodologies of classification of MBES backscatter data. This goal is attained through the achievement of specific objectives treated sequentially. First, the need for comparison is justified through a review of the diversity characterizing the fields of Benthic Habitat Mapping and Acoustic Seabed Classification, and of their use of MBES data products. Then, a case study is presented that compare the data products from a Kongsberg EM3000 MBES to the output map of an Acoustic Ground Discrimination Software based on data from a Single-beam Echosounder and to a Sidescan Sonar mosaic, in order to illustrate how map comparison measures could contribute to the comparison of these systems. Next, a number of measures for map-to-map comparison, inspired from the literature in land remote sensing, are presented, along with methodologies for their implementation in comparison of maps described with different legends. The benefit of these measures and methodologies is demonstrated through their application to maps obtained from the acoustic datasets presented previously. Finally, a more typical implementation of these measures is presented as a case study in which the development of two up-to-date classification methodologies of MBES backscatter data is complemented by the quantitative comparison of their output maps. In the process of developing and illustrating the use of methods for the assessment of map-to-map similarity, this thesis also presents methodologies for the processing and classification of backscatter data from MBES. In particular, the potential of the combined use of the spatial and angular information of these data for seabed classification is explored through the development of an original segmentation methodology that sequentially divides and aggregates segments defined from a MBES backscatter mosaic on the basis of their angular response content