Novel deep learning architectures for marine and aquaculture applications

Alzayat Saleh's research was in the area of artificial intelligence and machine learning to autonomously recognise fish and their morphological features from digital images. Here he created new deep learning architectures that solved various computer vision problems specific to the marine and aquaculture context. He found that these techniques can facilitate aquaculture management and environmental protection. Fisheries and conservation agencies can use his results for better monitoring strategies and sustainable fishing practices

Deep learning with self-supervision and uncertainty regularization to count fish in underwater images

Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the state-of-the-art method for many computer vision tasks, but it has yet to be properly explored to count animals. To this end, we employ deep learning, with a density-based regression approach, to count fish in low-resolution sonar images. We introduce a large dataset of sonar videos, deployed to record wild Lebranche mullet schools (Mugil liza), with a subset of 500 labelled images. We utilise abundant unlabelled data in a self-supervised task to improve the supervised counting task. For the first time in this context, by introducing uncertainty quantification, we improve model training and provide an accompanying measure of prediction uncertainty for more informed biological decision-making. Finally, we demonstrate the generalisability of our proposed counting framework through testing it on a recent benchmark dataset of high-resolution annotated underwater images from varying habitats (DeepFish). From experiments on both contrasting datasets, we demonstrate our network outperforms the few other deep learning models implemented for solving this task. By providing an open-source framework along with training data, our study puts forth an efficient deep learning template for crowd counting aquatic animals thereby contributing effective methods to assess natural populations from the ever-increasing visual data

Deep Semi-Supervised Semantic Segmentation in Multi-Frequency Echosounder Data

Multi-frequency echosounder data can provide a broad understanding of the underwater environment in a non-invasive manner. The analysis of echosounder data is, hence, a topic of great importance for the marine ecosystem. Semantic segmentation, a deep learning based analysis method predicting the class attribute of each acoustic intensity, has recently been in the spotlight of the fisheries and aquatic industry since its result can be used to estimate the abundance of the marine organisms. However, a fundamental problem with current methods is the massive reliance on the availability of large amounts of annotated training data, which can only be acquired through expensive handcrafted annotation processes, making such approaches unrealistic in practice. As a solution to this challenge, we propose a novel approach, where we leverage a small amount of annotated data (supervised deep learning) and a large amount of readily available unannotated data (unsupervised learning), yielding a new data-efficient and accurate semi-supervised semantic segmentation method, all embodied into a single end-to-end trainable convolutional neural networks architecture. Our method is evaluated on representative data from a sandeel survey in the North Sea conducted by the Norwegian Institute of Marine Research. The rigorous experiments validate that our method achieves comparable results utilizing only 40 percent of the annotated data on which the supervised method is trained, by leveraging unannotated data. The code is available at https://github.com/SFI-Visual-Intelligence/PredKlus-semisup-segmentation.Deep Semi-Supervised Semantic Segmentation in Multi-Frequency Echosounder DatasubmittedVersionsubmittedVersionsubmittedVersionacceptedVersio

Deep Semi-Supervised Semantic Segmentation in Multi-Frequency Echosounder Data

Multi-frequency echosounder data can provide a broad understanding of the underwater environment in a non-invasive manner. The analysis of echosounder data is, hence, a topic of great importance for the marine ecosystem. Semantic segmentation, a deep learning based analysis method predicting the class attribute of each acoustic intensity, has recently been in the spotlight of the fisheries and aquatic industry since its result can be used to estimate the abundance of the marine organisms. However, a fundamental problem with current methods is the massive reliance on the availability of large amounts of annotated training data, which can only be acquired through expensive handcrafted annotation processes, making such approaches unrealistic in practice. As a solution to this challenge, we propose a novel approach, where we leverage a small amount of annotated data (supervised deep learning) and a large amount of readily available unannotated data (unsupervised learning), yielding a new data-efficient and accurate semi-supervised semantic segmentation method, all embodied into a single end-to-end trainable convolutional neural networks architecture. Our method is evaluated on representative data from a sandeel survey in the North Sea conducted by the Norwegian Institute of Marine Research. The rigorous experiments validate that our method achieves comparable results utilizing only 40 percent of the annotated data on which the supervised method is trained, by leveraging unannotated data. The code is available at https://github.com/SFI-Visual-Intelligence/PredKlus-semisup-segmentation.Deep Semi-Supervised Semantic Segmentation in Multi-Frequency Echosounder DatasubmittedVersionsubmittedVersionsubmittedVersio

Machine Learning Methods with Noisy, Incomplete or Small Datasets

In many machine learning applications, available datasets are sometimes incomplete, noisy or affected by artifacts. In supervised scenarios, it could happen that label information has low quality, which might include unbalanced training sets, noisy labels and other problems. Moreover, in practice, it is very common that available data samples are not enough to derive useful supervised or unsupervised classifiers. All these issues are commonly referred to as the low-quality data problem. This book collects novel contributions on machine learning methods for low-quality datasets, to contribute to the dissemination of new ideas to solve this challenging problem, and to provide clear examples of application in real scenarios

Proceedings of the 8th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2023)

This volume gathers the papers presented at the Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023), Tampere, Finland, during 21–22 September 2023

Broadband Echosounder Calibration and Processing for Frequency Dependent Target Strength and Phase Measurements

An analysis technique is developed for the calibration and processing for the target strength and phase spectra using a broadband echosounder. A new variable ‘residual phase’ is introduced, which could be used as a target classifier. Implementation of the method to characterise marine organism from the open ocean, demonstrated consistent target strength and residual phase and the matching of both the variables to the output of the numerical scattering model verified the method

Broadband and statistical characterization of echoes from random scatterers : application to acoustic scattering by marine organisms

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution February 2013The interpretation of echoes collected by active remote-sensing systems, such as sonar and radar, is often ambiguous due to the complexities in the scattering processes involving the scatterers, the environment, and the sensing system. This thesis addresses this challenge using a combination of laboratory and fi eld experiments, theoretical modeling, and numerical simulations in the context of acoustic scattering by marine organisms. The unifying themes of the thesis are 1) quantitative characterization of the spectral, temporal, and statistical features derived from echoes collected using both broadband and narrowband signals, and 2) the interpretation of echoes by establishing explicit links between echo features and the sources of scattering through physics principles. This physics-based approach is distinct from the subjective descriptions and empirical methods employed in most conventional fisheries acoustic studies. The fi rst part focuses on understanding the dominant backscattering mechanisms of live squid as a function of orientation. The study provides the first broadband backscattering laboratory data set from live squid at all angles of orientation, and conclusively con firms the fluidlike, weakly-scattering material properties of squid through a series of detailed comparisons between data and predictions given by models derived based on the distorted-wave Born approximation. In the second part, an exact analytical narrowband model and a numerical broadband model are developed based on physics principles to describe the probability density function of the amplitudes of echo envelopes (echo pdf) of arbitrary aggregations of scatterers. The narrowband echo pdf model signi cantly outperforms the conventional mixture models in analyzing simulated mixed assemblages. When applied to analyze fish echoes collected in the ocean, the numerical density of sh estimated using the broadband echo pdf model is comparable to the density estimated using echo integration methods. These results demonstrate the power of the physics-based approach and give a rst-order assessment of the performance of echo statistics methods in echo interpretation. The new data, models, and approaches provided here are important for advancing the eld of active acoustic observation of the ocean.Taiwan Merit Scholarship (NSC-095-SAF-I-564-021-TMS), Office of Naval Research (ONR; grants N00014-10-1-0127, N00014-08-1-1162, N00014-07-1-1034), National Science Foundation (NSF; grant OCE-0928801), Naval Oceanographic Offi ce (grant N62306007-D9002), WHOI Ocean Life Institute, and the WHOI Academic Programs O ffice funds