Automatic plankton quantification using deep features

The study of marine plankton data is vital to monitor the health of the world’s oceans. In recent decades, automatic plankton recognition systems have proved useful to address the vast amount of data collected by specially engineered in situ digital imaging systems. At the beginning, these systems were developed and put into operation using traditional automatic classification techniques, which were fed with handdesigned local image descriptors (such as Fourier features), obtaining quite successful results. In the past few years, there have been many advances in the computer vision community with the rebirth of neural networks. In this paper, we leverage how descriptors computed using Convolutional Neural Networks (CNNs) trained with out-of-domain data are useful to replace hand-designed descriptors in the task of estimating the prevalence of each plankton class in a water sample. To achieve this goal, we have designed a broad set of experiments that show how effective these deep features are when working in combination with state-of-the-art quantification algorithms

In situ real-time Zooplankton Detection and Classification

Zooplankton plays a key-role on Earth’s ecosystem, emerging in the oceans and rivers in great quantities and diversity, making it an important and rather common topic on scientific studies. It serves as prey for many large living beings, such as fish and whales, and helps to keep the food chain stabilized by acting not only as prey to other animals but also as a consumer of phytoplankton, the main producers of oxygen on the planet. Zooplankton are also good indicators of environmental changes, such as global warming or rapid fluctuations in carbon dioxide in the atmosphere, since their abundance and existence is dependent on many environmental factors that indicate such changes. Not only is it important to study the numbers of zooplankton in the water masses, but also to know of what different species these numbers are composed of, as different species can provide information of different environmental attributes. In this thesis a possible solution for the zooplankton in situ detection and classification problem in real-time is proposed using a portable deep learning approach based on CNNs (Convolutional Neural Networks) deployed on INESC TEC’s MarinEye system. The proposed solution makes use of two different CNNs, one for the detection problem and another for the classification problem, running in MarinEye’s plankton imaging system, and portability is guaranteed by the use of the Movidius™ Neural Compute Stick as the deep learning motor in the hardware side. The software was implemented as a ROS node, which guarantees not only portability but facilitates communication between the imaging system and other MarinEye’s modules.O zooplâncton representa um papel fundamental no ecossistema do planeta, surgindo nos oceanos e rios em grandes quantidades numa elevada diversidade de espécies, sendo um objecto de estudo comum em publicações e artigos produzidos pela comunidade científica. A sua importância vem de entre outros factores do facto de ser a principal fonte de alimento de uma grande parte da vida marinha, desde pequenos peixes a baleias, e de ser um grande consumidor de fitoplâncton, a principal fonte de oxigénio do planeta. O zooplâncton é também um bom indicador de alterações ambientais, como o aquecimento global ou variações rápidas na quantidade de dióxido de carbono na atmosfera, uma vez que a sua abundância depende de diversos factores ambientais relacionados com tais mudanças, sendo não só importante perceber em que quantidades existe nas massas de água do planeta, mas também por que diferentes espécies está distribuído. Nesta tese é apresentada uma possível solução para a deteção e classificação de zooplâncton in situ e em tempo real, recorrendo a uma abordagem facilmente portável de Deep Learning, baseada em Redes Neuronais Convolucionais implementado no sistema MarinEye do INESC TEC. A solução proposta faz uso de duas arquitecturas de redes diferentes, uma dedicada à tarefa de deteção do zooplâncton, e outra dedicada `a sua classificação, implementadas no módulo de aquisição de imagens de plâncton do sistema MarinEye. A portabilidade e flexibilidade do sistema foi garantida através do uso da Movidius™ Neural Compute Stick como motor de deep learning, assim como da implementação do software como um nó de ROS, que garante não só a portabilidade do sistema, como também permite uma facilidade de comunicação entre os diferentes módulos do MarinEye

Deep learning for Plankton and Coral Classification

Oceans are the essential lifeblood of the Earth: they provide over 70% of the oxygen and over 97% of the water. Plankton and corals are two of the most fundamental components of ocean ecosystems, the former due to their function at many levels of the oceans food chain, the latter because they provide spawning and nursery grounds to many fish populations. Studying and monitoring plankton distribution and coral reefs is vital for environment protection. In the last years there has been a massive proliferation of digital imagery for the monitoring of underwater ecosystems and much research is concentrated on the automated recognition of plankton and corals. In this paper, we present a study about an automated system for monitoring of underwater ecosystems. The system here proposed is based on the fusion of different deep learning methods. We study how to create an ensemble based of different CNN models, fine tuned on several datasets with the aim of exploiting their diversity. The aim of our study is to experiment the possibility of fine-tuning pretrained CNN for underwater imagery analysis, the opportunity of using different datasets for pretraining models, the possibility to design an ensemble using the same architecture with small variations in the training procedure. The experimental results are very encouraging, our experiments performed on 5 well-knowns datasets (3 plankton and 2 coral datasets) show that the fusion of such different CNN models in a heterogeneous ensemble grants a substantial performance improvement with respect to other state-of-the-art approaches in all the tested problems. One of the main contributions of this work is a wide experimental evaluation of famous CNN architectures to report performance of both single CNN and ensemble of CNNs in different problems. Moreover, we show how to create an ensemble which improves the performance of the best single model

Applications of Machine Learning in Chemical and Biological Oceanography

Machine learning (ML) refers to computer algorithms that predict a meaningful output or categorize complex systems based on a large amount of data. ML is applied in various areas including natural science, engineering, space exploration, and even gaming development. This review focuses on the use of machine learning in the field of chemical and biological oceanography. In the prediction of global fixed nitrogen levels, partial carbon dioxide pressure, and other chemical properties, the application of ML is a promising tool. Machine learning is also utilized in the field of biological oceanography to detect planktonic forms from various images (i.e., microscopy, FlowCAM, and video recorders), spectrometers, and other signal processing techniques. Moreover, ML successfully classified the mammals using their acoustics, detecting endangered mammalian and fish species in a specific environment. Most importantly, using environmental data, the ML proved to be an effective method for predicting hypoxic conditions and harmful algal bloom events, an essential measurement in terms of environmental monitoring. Furthermore, machine learning was used to construct a number of databases for various species that will be useful to other researchers, and the creation of new algorithms will help the marine research community better comprehend the chemistry and biology of the ocean.Comment: 58 Pages, 5 Figure

Machine learning in marine ecology: an overview of techniques and applications

Machine learning covers a large set of algorithms that can be trained to identify patterns in data. Thanks to the increase in the amount of data and computing power available, it has become pervasive across scientific disciplines. We first highlight why machine learning is needed in marine ecology. Then we provide a quick primer on machine learning techniques and vocabulary. We built a database of ∼1000 publications that implement such techniques to analyse marine ecology data. For various data types (images, optical spectra, acoustics, omics, geolocations, biogeochemical profiles, and satellite imagery), we present a historical perspective on applications that proved influential, can serve as templates for new work, or represent the diversity of approaches. Then, we illustrate how machine learning can be used to better understand ecological systems, by combining various sources of marine data. Through this coverage of the literature, we demonstrate an increase in the proportion of marine ecology studies that use machine learning, the pervasiveness of images as a data source, the dominance of machine learning for classification-type problems, and a shift towards deep learning for all data types. This overview is meant to guide researchers who wish to apply machine learning methods to their marine datasets.Machine learning in marine ecology: an overview of techniques and applicationspublishedVersio

Vision-based techniques for automatic marine plankton classification

Plankton are an important component of life on Earth. Since the 19th century, scientists have attempted to quantify species distributions using many techniques, such as direct counting, sizing, and classification with microscopes. Since then, extraordinary work has been performed regarding the development of plankton imaging systems, producing a massive backlog of images that await classification. Automatic image processing and classification approaches are opening new avenues for avoiding time-consuming manual procedures. While some algorithms have been adapted from many other applications for use with plankton, other exciting techniques have been developed exclusively for this issue. Achieving higher accuracy than that of human taxonomists is not yet possible, but an expeditious analysis is essential for discovering the world beyond plankton. Recent studies have shown the imminent development of real-time, in situ plankton image classification systems, which have only been slowed down by the complex implementations of algorithms on low-power processing hardware. This article compiles the techniques that have been proposed for classifying marine plankton, focusing on automatic methods that utilize image processing, from the beginnings of this field to the present day.Funding for open access charge: Universidad de Málaga / CBUA. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The authors wish to thank Alonso Hernández-Guerra for his frm support in the development of oceanographic technology. Special thanks to Laia Armengol for her help in the domain of plankton. This study has been funded by Feder of the UE through the RES-COAST Mac-Interreg pro ject (MAC2/3.5b/314). We also acknowledge the European Union projects SUMMER (Grant Agreement 817806) and TRIATLAS (Grant Agreement 817578) from the Horizon 2020 Research and Innovation Programme and the Ministry of Science from the Spanish Government through the Project DESAFÍO (PID2020-118118RB-I00)

A Novel Detection Refinement Technique for Accurate Identification of Nephrops norvegicus Burrows in Underwater Imagery

With the evolution of the convolutional neural network (CNN), object detection in the underwater environment has gained a lot of attention. However, due to the complex nature of the underwater environment, generic CNN-based object detectors still face challenges in underwater object detection. These challenges include image blurring, texture distortion, color shift, and scale variation, which result in low precision and recall rates. To tackle this challenge, we propose a detection refinement algorithm based on spatial–temporal analysis to improve the performance of generic detectors by suppressing the false positives and recovering the missed detections in underwater videos. In the proposed work, we use state-of-the-art deep neural networks such as Inception, ResNet50, and ResNet101 to automatically classify and detect the Norway lobster Nephrops norvegicus burrows from underwater videos. Nephrops is one of the most important commercial species in Northeast Atlantic waters, and it lives in burrow systems that it builds itself on muddy bottoms. To evaluate the performance of proposed framework, we collected the data from the Gulf of Cadiz. From experiment results, we demonstrate that the proposed framework effectively suppresses false positives and recovers missed detections obtained from generic detectors. The mean average precision (mAP) gained a 10% increase with the proposed refinement technique.Versión del edito

PORIFERAL VISION: Deep Transfer Learning-based Sponge Spicules Identification & Taxonomic Classification

The phylum Porifera includes the aquatic organisms known as sponges. Sponges are classified into four classes: Calcarea, Hexactinellida, Demospongiae, and Homoscleromorpha. Within Demospongiae and Hexactinellida, sponges’ skeletons are needle-like spicules made of silica. With a wide variety of shapes and sizes, these siliceous spicules’ morphology plays a pivotal role in assessing and understanding sponges\u27 taxonomic diversity and evolution. In marine ecosystems, when sponges die their bodies disintegrate over time, but their spicules remain in the sediments as fossilized records that bear ample taxonomic information to reconstruct the evolution of sponge communities and sponge phylogeny. Traditional methods of identifying spicules from core samples of marine sediments are labor-intensive and cannot scale to the scope needed for large analysis. Through the incorporation of high-throughput microscopy and deep learning, image classification has made significant strides toward automating the task of species recognition and taxonomic classification. Even with sparse training data and highly specific image domains, deep convolutional neural networks (DCNNs) were able to extract taxonomic features among morphologically diverse microfossils. Using transfer learning, training a classifier on pretrained DCNNs has achieved recent successes in classifying similar microfossils, such as diatom frustules and radiolarian skeletons. In this project, I address the reliability of pretrained models to perform spicule identification and class-level classification. Using FlowCam technology to photograph individual microparticles, our dataset consists of spicule and non-spicule types without additional image segmentation and augmentation. Our proposed method is a pre-trained model with a custom classifier that performs two different binary classifications: a spicule vs non-spicule classification, and a taxonomic classification of Demospongiae vs. Hexactinellida. We evaluate the effect of implementing different DCNN architectures, data set sizes, and classifiers on image classification performance. Surprisingly, MobileNet, a relatively new and small architecture, showed the best performance while still being the most computationally efficient. Other studies that didn’t involve MobileNet had similar high accuracies for multi-class classifications with fewer training images. The reliability of DCNNs for binary spicule classification implicates the promising approach of a more nuanced multi-class/taxonomic classification. Future work should build multi-class classification that ranges more biogenic materials for the identification or more sponge taxonomic levels for species classification