Deep CNN and MLP-based vision systems for algae detection in automatic inspection of underwater pipelines

Artificial neural networks, such as the multilayer perceptron (MLP), have been increasingly employed in various applications. Recently, deep neural networks, specially convolutional neural networks (CNN), have received considerable attention due to their ability to extract and represent high-level abstractions in data sets. This work describes a vision inspection system based on deep learning and computer vision algorithms for detection of algae in underwater pipelines. The proposed algorithm comprises a CNN or a MLP network, followed by a post-processing stage operating in spatial and temporal domains, employing clustering of neighboring detection positions and a region interception framebuffer. The performances of MLP, employing different descriptors, and CNN classifiers are compared in real-world scenarios. It is shown that the post-processing stage considerably decreases the number of false positives, resulting in an accuracy rate of 99.39%.Redes neurais artificiais, como o perceptron multicamada (MLP), têm sido cada vez mais empregadas em várias aplicações. Recentemente, as redes neurais profundas (deep neural networks), especialmente as redes neurais convolutivas (CNN), receberam atenção considerável devido à sua capacidade de extrair e representar abstrações de alto nível em conjuntos de dados. Esta dissertação descreve um sistema de inspeção automático baseado em algoritmos de aprendizado profundo (deep learning) e visão computacional para detecção de algas em dutos submarinos. O algoritmo proposto compreende uma rede CNN ou MLP, seguida de uma fase de pós-processamento que opera em domínios espaciais e temporais, empregando agrupamento de posições de detecção vizinhas e um buffer das regiões de interseção ao longo dos quadros. Os desempenhos de MLP, empregando diferentes descritores, e os classificadores CNN são comparados em cenários do mundo real. Mostra-se que a fase de pos-processamento diminui consideravelmente o número de falsos positivos, resultando em uma taxa de acerto de 99,39%

Classification of underwater pipeline events using deep convolutional neural networks

Automatic inspection of underwater pipelines has been a task of growing importance for the detection of four different types of events: inner coating exposure, presence of algae, flanges and concrete blankets. Such inspections might benefit of machine learning techniques in order to accurately classify such occurrences. In this work, we present a deep convolutional neural network algorithm for the classification of underwater pipeline events. The neural network architecture and parameters that result in optimal classifier performance are selected. The convolutional neural network technique outperforms the perceptron algorithm preceded by wavelet feature extraction for different event classes, reaching on average 93.2% classification accuracy, while the accuracy achieved by the perceptron is 91.2%. Besides the results obtained in the test set, accuracy and cross entropy curves obtained in the validation set during training are analyzed, so that the performances of each method and for each event class are compared. Visualizations of the convolutional neural network intermediate layer outputs are also provided. These visualizations are interpreted and associated to the results obtained.A inspeção automática de dutos submarinos tem sido uma tarefa de crescente importância para a detecção de diferentes tipos de eventos, dos quais destacam-se armadura exposta, presença de algas, flanges e manta. Tais inspeções podem se beneficiar de técnicas de aprendizado de máquinas para classificar acuradamente essas ocorrências. Neste trabalho, apresenta-se um algoritmo de redes neurais convolucionais para classificação de eventos em dutos submarinos. A arquitetura e os parâmetros da rede neural que resultam em desempenho de classificação ótimo são selecionados. A técnica de rede neural convolucional, em comparação ao algoritmo do perceptron precedido por extração de features wavelet, apresenta desempenho superior para diferentes classes de eventos, alcançando em média acurácia de classificação de 93.2%, enquanto o desempenho alcançado pelo perceptron é de 91.2%. Além dos resultados obtidos no conjunto de teste, são analisadas as curvas de acurácia e de entropia cruzada obtidas para o conjunto de validação ao longo do treinamento, de modo a comparar os desempenhos de cada método e para cada classe de eventos. São também fornecidas visualizações das saídas das camadas intermediárias da rede convolucional. Essas visualizações são interpretadas e associadas aos resultados obtidos

Automatic Annotation of Subsea Pipelines using Deep Learning

Regulatory requirements for sub-sea oil and gas operators mandates the frequent inspection of pipeline assets to ensure that their degradation and damage are maintained at acceptable levels. The inspection process is usually sub-contracted to surveyors who utilize sub-sea Remotely Operated Vehicles (ROVs), launched from a surface vessel and piloted over the pipeline. ROVs capture data from various sensors/instruments which are subsequently reviewed and interpreted by human operators, creating a log of event annotations; a slow, labor-intensive and costly process. The paper presents an automatic image annotation framework that identifies/classifies key events of interest in the video footage viz. exposure, burial, field joints, anodes, and free spans. The reported methodology utilizes transfer learning with a Deep Convolutional Neural Network (ResNet-50), fine-tuned on real-life, representative data from challenging sub-sea environments with low lighting conditions, sand agitation, sea-life and vegetation. The network outputs are configured to perform multi-label image classifications for critical events. The annotation performance varies between 95.1 and 99.7 in terms of accuracy and 90.4 and 99.4 in terms of F1-Score depending on event type. The performance results are on a per-frame basis and corroborate the potential of the algorithm to be the foundation for an intelligent decision support framework that automates the annotation process. The solution can execute annotations in real-time and is significantly more cost-effective than human-only approaches

A comparison of the performance of 2D and 3D convolutional neural networks for subsea survey video classification

Utilising deep learning image classification to auto-matically annotate subsea pipeline video surveys can facilitate the tedious and labour-intensive process, resulting in significant time and cost savings. However, the classification of events on subsea survey videos (frame sequences) by models trained on individual frames have been proven to vary, leading to inaccuracies. The paper extends previous work on the automatic annotation of individual subsea survey frames by comparing the performance of 2D and 3D Convolutional Neural Networks (CNNs) in classifying frame sequences. The study explores the classification of burial, exposure, free span, field joint, and anode events. Sampling and regularization techniques are designed to address the challenges of an underwater inspection video dataset owing to the environment. Results show that a 2D CNN with rolling average can outperform a 3D CNN, achieving an Exact Match Ratio of 85% and F1-Score of 90%, whilst being more computationally efficient

Internet of Underwater Things and Big Marine Data Analytics -- A Comprehensive Survey

The Internet of Underwater Things (IoUT) is an emerging communication ecosystem developed for connecting underwater objects in maritime and underwater environments. The IoUT technology is intricately linked with intelligent boats and ships, smart shores and oceans, automatic marine transportations, positioning and navigation, underwater exploration, disaster prediction and prevention, as well as with intelligent monitoring and security. The IoUT has an influence at various scales ranging from a small scientific observatory, to a midsized harbor, and to covering global oceanic trade. The network architecture of IoUT is intrinsically heterogeneous and should be sufficiently resilient to operate in harsh environments. This creates major challenges in terms of underwater communications, whilst relying on limited energy resources. Additionally, the volume, velocity, and variety of data produced by sensors, hydrophones, and cameras in IoUT is enormous, giving rise to the concept of Big Marine Data (BMD), which has its own processing challenges. Hence, conventional data processing techniques will falter, and bespoke Machine Learning (ML) solutions have to be employed for automatically learning the specific BMD behavior and features facilitating knowledge extraction and decision support. The motivation of this paper is to comprehensively survey the IoUT, BMD, and their synthesis. It also aims for exploring the nexus of BMD with ML. We set out from underwater data collection and then discuss the family of IoUT data communication techniques with an emphasis on the state-of-the-art research challenges. We then review the suite of ML solutions suitable for BMD handling and analytics. We treat the subject deductively from an educational perspective, critically appraising the material surveyed.Comment: 54 pages, 11 figures, 19 tables, IEEE Communications Surveys & Tutorials, peer-reviewed academic journa

Hydroacoustic Mapping of Geogenic Hard Substrates: Challenges and Review of German Approaches

Subtidal hard substrate habitats are unique habitats in the marine environment. They provide crucial ecosystem services that are socially relevant, such as water clearance or as nursery space for fishes. With increasing marine usage and changing environmental conditions, pressure on reefs is increasing. All relevant directives and conventions around Europe include sublittoral hard substrate habitats in any manner. However, detailed specifications and specific advices about acquisition or delineation of these habitats are internationally rare although the demand for single object detection for e.g., ensuring safe navigation or to understand ecosystem functioning is increasing. To figure out the needs for area wide hard substrate mapping supported by automatic detection routines this paper reviews existing delineation rules and definitions relevant for hard substrate mapping. We focus on progress reached in German approval process resulting in first hydroacoustic mapping advices. In detail, we summarize present knowledge of hard substrate occurrence in the German North Sea and Baltic Sea, describes the development of hard substrate investigations and state of the art mapping techniques as well as automated analysis routines

Localization, Mapping and SLAM in Marine and Underwater Environments

The use of robots in marine and underwater applications is growing rapidly. These applications share the common requirement of modeling the environment and estimating the robots’ pose. Although there are several mapping, SLAM, target detection and localization methods, marine and underwater environments have several challenging characteristics, such as poor visibility, water currents, communication issues, sonar inaccuracies or unstructured environments, that have to be considered. The purpose of this Special Issue is to present the current research trends in the topics of underwater localization, mapping, SLAM, and target detection and localization. To this end, we have collected seven articles from leading researchers in the field, and present the different approaches and methods currently being investigated to improve the performance of underwater robots

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Deep learning for internet of underwater things and ocean data analytics

The Internet of Underwater Things (IoUT) is an emerging technological ecosystem developed for connecting objects in maritime and underwater environments. IoUT technologies are empowered by an extreme number of deployed sensors and actuators. In this thesis, multiple IoUT sensory data are augmented with machine intelligence for forecasting purposes

INTEROPERABILITY FOR MODELING AND SIMULATION IN MARITIME EXTENDED FRAMEWORK

This thesis reports on the most relevant researches performed during the years of the Ph.D. at the Genova University and within the Simulation Team. The researches have been performed according to M&S well known recognized standards. The studies performed on interoperable simulation cover all the environments of the Extended Maritime Framework, namely Sea Surface, Underwater, Air, Coast & Land, Space and Cyber Space. The applications cover both the civil and defence domain. The aim is to demonstrate the potential of M&S applications for the Extended Maritime Framework, applied to innovative unmanned vehicles as well as to traditional assets, human personnel included. A variety of techniques and methodology have been fruitfully applied in the researches, ranging from interoperable simulation, discrete event simulation, stochastic simulation, artificial intelligence, decision support system and even human behaviour modelling