Automatic plankton quantification using deep features

The study of marine plankton data is vital to monitor the health of the world’s oceans. In recent decades, automatic plankton recognition systems have proved useful to address the vast amount of data collected by specially engineered in situ digital imaging systems. At the beginning, these systems were developed and put into operation using traditional automatic classification techniques, which were fed with handdesigned local image descriptors (such as Fourier features), obtaining quite successful results. In the past few years, there have been many advances in the computer vision community with the rebirth of neural networks. In this paper, we leverage how descriptors computed using Convolutional Neural Networks (CNNs) trained with out-of-domain data are useful to replace hand-designed descriptors in the task of estimating the prevalence of each plankton class in a water sample. To achieve this goal, we have designed a broad set of experiments that show how effective these deep features are when working in combination with state-of-the-art quantification algorithms

A holographic system for subsea recording and analysis of plankton and other marine particles

We report here details of the design, development, initial testing and field-deployment of the HOLOMAR system for in-situ subsea holography and analysis of marine plankton and nonliving particles. HOLOMAR comprises a submersible holographic camera ("HoloCam") able to record in-line and off-axis holograms at depths down to 100 m, together with specialised reconstruction hardware ("HoloScan") linked to custom image processing and classification software. The HoloCam consists of a laser and power supply, holographic recording optics and holographic plate holders, a water-tight housing and a support frame. It utilises two basic holographic geometries, in-line and off-axis such that a wide range of species, sizes and concentrations can be recorded. After holograms have been recorded and processed they are reconstructed in full three-dimensional detail in air in a dedicated replay facility. A computer-controlled microscope, using video cameras to record the image at a given depth, is used to digitise the scene. Specially written software extracts a binarised image of an object in its true focal plane and is classified using a neural network. The HoloCam was deployed on two separate cruises in a Scottish sea loch (Loch Etive) to a depth of 100 m and over 300 holograms were recorded

Application of statistical learning theory to plankton image analysis

Submitted to the Joint Program in Applied Ocean Science and Engineering in partial fulfillment of the requirements for the degree of Doctor of Philosophy At the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2006A fundamental problem in limnology and oceanography is the inability to quickly identify and map distributions of plankton. This thesis addresses the problem by applying statistical machine learning to video images collected by an optical sampler, the Video Plankton Recorder (VPR). The research is focused on development of a real-time automatic plankton recognition system to estimate plankton abundance. The system includes four major components: pattern representation/feature measurement, feature extraction/selection, classification, and abundance estimation. After an extensive study on a traditional learning vector quantization (LVQ) neural network (NN) classifier built on shape-based features and different pattern representation methods, I developed a classification system combined multi-scale cooccurrence matrices feature with support vector machine classifier. This new method outperforms the traditional shape-based-NN classifier method by 12% in classification accuracy. Subsequent plankton abundance estimates are improved in the regions of low relative abundance by more than 50%. Both the NN and SVM classifiers have no rejection metrics. In this thesis, two rejection metrics were developed. One was based on the Euclidean distance in the feature space for NN classifier. The other used dual classifier (NN and SVM) voting as output. Using the dual-classification method alone yields almost as good abundance estimation as human labeling on a test-bed of real world data. However, the distance rejection metric for NN classifier might be more useful when the training samples are not “good” ie, representative of the field data. In summary, this thesis advances the current state-of-the-art plankton recognition system by demonstrating multi-scale texture-based features are more suitable for classifying field-collected images. The system was verified on a very large realworld dataset in systematic way for the first time. The accomplishments include developing a multi-scale occurrence matrices and support vector machine system, a dual-classification system, automatic correction in abundance estimation, and ability to get accurate abundance estimation from real-time automatic classification. The methods developed are generic and are likely to work on range of other image classification applications.This work was supported by National Science Foundation Grants OCE-9820099 and Woods Hole Oceanographic Institution academic program

In situ real-time Zooplankton Detection and Classification

Zooplankton plays a key-role on Earth’s ecosystem, emerging in the oceans and rivers in great quantities and diversity, making it an important and rather common topic on scientific studies. It serves as prey for many large living beings, such as fish and whales, and helps to keep the food chain stabilized by acting not only as prey to other animals but also as a consumer of phytoplankton, the main producers of oxygen on the planet. Zooplankton are also good indicators of environmental changes, such as global warming or rapid fluctuations in carbon dioxide in the atmosphere, since their abundance and existence is dependent on many environmental factors that indicate such changes. Not only is it important to study the numbers of zooplankton in the water masses, but also to know of what different species these numbers are composed of, as different species can provide information of different environmental attributes. In this thesis a possible solution for the zooplankton in situ detection and classification problem in real-time is proposed using a portable deep learning approach based on CNNs (Convolutional Neural Networks) deployed on INESC TEC’s MarinEye system. The proposed solution makes use of two different CNNs, one for the detection problem and another for the classification problem, running in MarinEye’s plankton imaging system, and portability is guaranteed by the use of the Movidius™ Neural Compute Stick as the deep learning motor in the hardware side. The software was implemented as a ROS node, which guarantees not only portability but facilitates communication between the imaging system and other MarinEye’s modules.O zooplâncton representa um papel fundamental no ecossistema do planeta, surgindo nos oceanos e rios em grandes quantidades numa elevada diversidade de espécies, sendo um objecto de estudo comum em publicações e artigos produzidos pela comunidade científica. A sua importância vem de entre outros factores do facto de ser a principal fonte de alimento de uma grande parte da vida marinha, desde pequenos peixes a baleias, e de ser um grande consumidor de fitoplâncton, a principal fonte de oxigénio do planeta. O zooplâncton é também um bom indicador de alterações ambientais, como o aquecimento global ou variações rápidas na quantidade de dióxido de carbono na atmosfera, uma vez que a sua abundância depende de diversos factores ambientais relacionados com tais mudanças, sendo não só importante perceber em que quantidades existe nas massas de água do planeta, mas também por que diferentes espécies está distribuído. Nesta tese é apresentada uma possível solução para a deteção e classificação de zooplâncton in situ e em tempo real, recorrendo a uma abordagem facilmente portável de Deep Learning, baseada em Redes Neuronais Convolucionais implementado no sistema MarinEye do INESC TEC. A solução proposta faz uso de duas arquitecturas de redes diferentes, uma dedicada à tarefa de deteção do zooplâncton, e outra dedicada `a sua classificação, implementadas no módulo de aquisição de imagens de plâncton do sistema MarinEye. A portabilidade e flexibilidade do sistema foi garantida através do uso da Movidius™ Neural Compute Stick como motor de deep learning, assim como da implementação do software como um nó de ROS, que garante não só a portabilidade do sistema, como também permite uma facilidade de comunicação entre os diferentes módulos do MarinEye

Vision-based techniques for automatic marine plankton classification

Plankton are an important component of life on Earth. Since the 19th century, scientists have attempted to quantify species distributions using many techniques, such as direct counting, sizing, and classification with microscopes. Since then, extraordinary work has been performed regarding the development of plankton imaging systems, producing a massive backlog of images that await classification. Automatic image processing and classification approaches are opening new avenues for avoiding time-consuming manual procedures. While some algorithms have been adapted from many other applications for use with plankton, other exciting techniques have been developed exclusively for this issue. Achieving higher accuracy than that of human taxonomists is not yet possible, but an expeditious analysis is essential for discovering the world beyond plankton. Recent studies have shown the imminent development of real-time, in situ plankton image classification systems, which have only been slowed down by the complex implementations of algorithms on low-power processing hardware. This article compiles the techniques that have been proposed for classifying marine plankton, focusing on automatic methods that utilize image processing, from the beginnings of this field to the present day.Funding for open access charge: Universidad de Málaga / CBUA. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The authors wish to thank Alonso Hernández-Guerra for his frm support in the development of oceanographic technology. Special thanks to Laia Armengol for her help in the domain of plankton. This study has been funded by Feder of the UE through the RES-COAST Mac-Interreg pro ject (MAC2/3.5b/314). We also acknowledge the European Union projects SUMMER (Grant Agreement 817806) and TRIATLAS (Grant Agreement 817578) from the Horizon 2020 Research and Innovation Programme and the Ministry of Science from the Spanish Government through the Project DESAFÍO (PID2020-118118RB-I00)

DIGITAL IMAGE IDENTIFICATION OF PLANKTON USING REGIONPROPS AND BAGGING DECISION TREE ALGORITHM

Peranan plankton sangat penting bagi kehidupan organisme disekitarnya, sehingga penelitian prihal plankton sangatlah dibutuhkan karena kaitannya dengan kelangsungan kehidupan mahluk hidup lainnya.              Kendala yang sering didapatkan dalam hal penelitian plankton khususnya dalam hal pengidentifikasian plankton yaitu tidak efisiennya dalam aspek waktu dan organisme ini memiliki ukuran rata-rata yang sangat kecil. Dalam hal ini diperlukan alternatif yang lebih baik dalam pengidentifikasian jenis plankton ini dengan cara pemrosesan gambar pada citra plankton secara digital atau biasa disebut dengan istilah “Digital Image Processing”. Penelitian ini bertujuan untuk melakukan pengolahan citra digital plankton sebanyak 144 citra yang yang dibagi menjadi 75% sebagai data pelatihan dan 25% sebagai data pengujian, dan citra tersebut didapatkan dari riset pada yayasan Kanopi Indonesia. Dalam prosesnya citra ini dianalisa bentuk menggunakan fungsi Regionprops sehingga didapatkan fitur pembeda dari masing-masing jenis plankton. Setelah citra terekstraksi fitur nya selanjutnya dilakukan pengolahan data dengan mengklasifikasikan setiap jenis plankton tersebut. Untuk menghasilkan sebuah klasifikasi data yang lebih baik, dalam penelitian ini menggunakan algoritma Bagging Decision Tree dalam pengolahan data nya dan menghasilkan akurasi sebesar 92.59%. Algoritma Bagging Decision Tree ini cukup baik dan mudah untuk di implemntasikan kedalam sebuah program identifikasi jenis plankton, terbukti dengan pengujian pada data citra pengujian menghasilkan 33 citra teridentifikasi dengan benar dari total pengujian sebanyak 36 citra