Fully automatic detection and classification of phytoplankton specimens in digital microscopy images

©2021. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/. This version of the article Rivas-Villar, D., Rouco, J., Carballeira, R., Penedo, M. G., & Novo, J. (2021). “Fully automatic detection and classification of phytoplankton specimens in digital microscopy images” has been accepted for publication in Computer Methods and Programs in Biomedicine, 200(105923), 105923. The Version of Record is available online at: https://doi.org/10.1016/j.cmpb.2020.105923.[Abstract]: Background and objective: The proliferation of toxin-producing phytoplankton species can compromise the quality of the water sources. This contamination is difficult to detect, and consequently to be neutralised, since normal water purification techniques are ineffective. Currently, the water analyses about phytoplankton are commonly performed by the specialists with manual routine analyses, which represents a major limitation. The adequate identification and classification of phytoplankton specimens requires intensive training and expertise. Additionally, the performed analysis involves a lengthy process that exhibits serious problems of reliability and repeatability as inter-expert agreement is not always reached. Considering all those factors, the automatization of these analyses is, therefore, highly desirable to reduce the workload of the specialists and facilitate the process. Methods: This manuscript proposes a novel fully automatic methodology to perform phytoplankton analyses in digital microscopy images of water samples taken with a regular light microscope. In particular, we propose a method capable of analysing multi-specimen images acquired using a simplified systematic protocol. In contrast with prior approaches, this enables its use without the necessity of an expert taxonomist operating the microscope. The system is able to detect and segment the different existing phytoplankton specimens, with high variability in terms of visual appearances, and to merge them into colonies and sparse specimens when necessary. Moreover, the system is capable of differentiating them from other similar objects like zooplankton, detritus or mineral particles, among others, and then classify the specimens into defined target species of interest using a machine learning-based approach. Results: The proposed system provided satisfactory and accurate results in every step. The detection step provided a FNR of 0.4%. Phytoplankton detection, that is, differentiating true phytoplankton from similar objects (zooplankton, minerals, etc.), provided a result of 84.07% of precision at 90% of recall. The target species classification, reported an overall accuracy of 87.50%. The recall levels for each species are, 81.82% for W. naegeliana, 57.15% for A. spiroides, 85.71% for D. sociale and 95% for the ”Other” group, a set of relevant toxic and interesting species widely spread over the samples. Conclusions: The proposed methodology provided accurate results in all the designed steps given the complexity of the problem, particularly in terms of specimen identification, phytoplankton differentiation as well as the classification of the defined target species. Therefore, this fully automatic system represents a robust and consistent tool to aid the specialists in the analysis of the quality of the water sources and potability.This work is supported by the European Regional Development Fund (ERDF) of the European Union and Xunta de Galicia through Centro de Investigación del Sistema Universitario de Galicia, ref. ED431G 2019/01.Xunta de Galicia; ED431G 2019/0

Automatic Detection of Freshwater Phytoplankton Specimens in Conventional Microscopy Images

[Abstract] Water safety and quality can be compromised by the proliferation of toxin-producing phytoplankton species, requiring continuous monitoring of water sources. This analysis involves the identification and counting of these species which requires broad experience and knowledge. The automatization of these tasks is highly desirable as it would release the experts from tedious work, eliminate subjective factors, and improve repeatability. Thus, in this preliminary work, we propose to advance towards an automatic methodology for phytoplankton analysis in digital images of water samples acquired using regular microscopes. In particular, we propose a novel and fully automatic method to detect and segment the existent phytoplankton specimens in these images using classical computer vision algorithms. The proposed method is able to correctly detect sparse colonies as single phytoplankton candidates, thanks to a novel fusion algorithm, and is able to differentiate phytoplankton specimens from other image objects in the microscope samples (like minerals, bubbles or detritus) using a machine learning based approach that exploits texture and colour features. Our preliminary experiments demonstrate that the proposed method provides satisfactory and accurate results.This work is supported by the European Regional Development Fund (ERDF) of the European Union and Xunta de Galicia through Centro de Investigación del Sistema Universitario de Galicia, ref. ED431G 2019/01Xunta de Galicia; ED431G 2019/0

Efficient Unsupervised Learning for Plankton Images

Monitoring plankton populations in situ is fundamental to preserve the aquatic ecosystem. Plankton microorganisms are in fact susceptible of minor environmental perturbations, that can reflect into consequent morphological and dynamical modifications. Nowadays, the availability of advanced automatic or semi-automatic acquisition systems has been allowing the production of an increasingly large amount of plankton image data. The adoption of machine learning algorithms to classify such data may be affected by the significant cost of manual annotation, due to both the huge quantity of acquired data and the numerosity of plankton species. To address these challenges, we propose an efficient unsupervised learning pipeline to provide accurate classification of plankton microorganisms. We build a set of image descriptors exploiting a two-step procedure. First, a Variational Autoencoder (VAE) is trained on features extracted by a pre-trained neural network. We then use the learnt latent space as image descriptor for clustering. We compare our method with state-of-the-art unsupervised approaches, where a set of pre-defined hand-crafted features is used for clustering of plankton images. The proposed pipeline outperforms the benchmark algorithms for all the plankton datasets included in our analysis, providing better image embedding properties

Imaging-in-flow: digital holographic microscopy as a novel tool to detect and classify nanoplanktonic organisms

Traditional taxonomic identification of planktonic organisms is based on light microscopy, which is both time-consuming and tedious. In response, novel ways of automated (machine) identification, such as flow cytometry, have been investigated over the last two decades. To improve the taxonomic resolution of particle analysis, recent developments have focused on "imaging-in-flow," i.e., the ability to acquire microscopic images of planktonic cells in a flow-through mode. Imaging-in-flow systems are traditionally based on classical brightfield microscopy and are faced with a number of issues that decrease the classification performance and accuracy (e. g., projection variance of cells, migration of cells out of the focus plane). Here, we demonstrate that a combination of digital holographic microscopy (DHM) with imaging-in-flow can improve the detection and classification of planktonic organisms. In addition to light intensity information, DHM provides quantitative phase information, which generates an additional and independent set of features that can be used in classification algorithms. Moreover, the capability of digitally refocusing greatly increases the depth of field, enables a more accurate focusing of cells, and reduces the effects of position variance. Nanoplanktonic organisms similar in shape were successfully classified from images captured with an off-axis DHM with partial coherence. Textural features based on DHM phase information proved more efficient in separating the three tested phytoplankton species compared with shape-based features or textural features based on light intensity. An overall classification score of 92.4% demonstrates the potential of holographic-based imaging-in-flow for similar looking organisms in the nanoplankton range

Morphological bases of phytoplankton energy management and physiological responses unveiled by 3D subcellular imaging

Eukaryotic phytoplankton have a small global biomass but play major roles in primary production and climate. Despite improved understanding of phytoplankton diversity and evolution, we largely ignore the cellular bases of their environmental plasticity. By comparative 3D morphometric analysis across seven distant phytoplankton taxa, we observe constant volume occupancy by the main organelles and preserved volumetric ratios between plastids and mitochondria. We hypothesise that phytoplankton subcellular topology is modulated by energy-management constraints. Consistent with this, shifting the diatom Phaeodactylum from low to high light enhances photosynthesis and respiration, increases cell-volume occupancy by mitochondria and the plastid CO2-fixing pyrenoid, and boosts plastid mitochondria contacts. Changes in organelle architectures and interactions also accompany Nannochloropsis acclimation to different trophic lifestyles, along with respiratory and photosynthetic responses. By revealing evolutionarily-conserved topologies of energy-managing organelles, and their role in phytoplankton acclimation, this work deciphers phytoplankton responses at subcellular scales

Quantitative comparison of taxa and taxon concepts in the diatom genus <i>Fragilariopsis</i>: a case study on using slide scanning, multiexpert image annotation, and image analysis in taxonomy

Semiautomated methods for microscopic image acquisition, image analysis, and taxonomic identification have repeatedly received attention in diatom analysis. Less well studied is the question whether and how such methods might prove useful for clarifying the delimitation of species that are difficult to separate for human taxonomists. To try to answer this question, three very similar Fragilariopsis species endemic to the Southern Ocean were targeted in this study: F. obliquecostata, F. ritscheri, and F. sublinearis. A set of 501 extended focus depth specimen images were obtained using a standardized, semiautomated microscopic procedure. Twelve diatomists independently identified these specimen images in order to reconcile taxonomic opinions and agree upon a taxonomic gold standard. Using image analyses, we then extracted morphometric features representing taxonomic characters of the target taxa. The discriminating ability of individual morphometric features was tested visually and statistically, and multivariate classification experiments were performed to test the agreement of the quantitatively defined taxa assignments with expert consensus opinion. Beyond an updated differential diagnosis of the studied taxa, our study also shows that automated imaging and image analysis procedures for diatoms are coming close to reaching a broad applicability for routine use.Facultad de Ciencias Naturales y Muse

Quantitative comparison of taxa and taxon concepts in the diatom genus Fragilariopsis: a case study on using slide scanning, multi‐expert image annotation and image analysis in taxonomy

Semi‐automated methods for microscopic image acquisition, image analysis and taxonomic identification have repeatedly received attention in diatom analysis. Less well studied is the question whether and how such methods might prove useful for clarifying the delimitation of species that are difficult to separate for human taxonomists. To try to answer this question, three very similar Fragilariopsis species endemic to the Southern Ocean were targeted in this study: F. obliquecostata, F. ritscheri, and F. sublinearis. A set of 501 extended focus depth specimen images were obtained using a standardized, semi‐automated microscopic procedure. Twelve diatomists independently identified these specimen images in order to reconcile taxonomic opinions and agree upon a taxonomic gold standard. Using image analyses, we then extracted morphometric features representing taxonomic characters of the target taxa. The discriminating ability of individual morphometric features was tested visually and statistically, and multivariate classification experiments were performed to test the agreement of the quantitatively‐defined taxa assignments with expert consensus opinion. Beyond an updated differential diagnosis of the studied taxa, our study also shows that automated imaging and image analysis procedures for diatoms are coming close to reaching a broad applicability for routine use

Morphometrics of Southern Ocean diatoms using high throughput imaging and semi-automated image analysis

Since the ADIAC project, which ended more than 15 years ago, not much progress in automating morphometric analysis of diatoms from slide-mounted material has been published, and no ready-to-use system has become available. This thesis work is the first to implement such a system completely, covering all aspects of the underlying imaging and image processing pipeline, by combining a commercially available slide scanning microscope with my diatom morphometry software SHERPA. I was able to show the applicability as well as the potential of this approach by executing a series of smaller and two large-scale morphometry projects. The extensive sampling sizes, which were made possible only by the new workflow, enabled the first observations of life cycle related size distribution changes of Fragilariopsis kerguelensis in its natural habitat, leading to hypotheses on influences of reproduction, grazing and environmental changes in one of the most important diatom species of the Southern Ocean. In a second large-scale investigation, SHERPA's precise morphometric measurements revealed a second F. kerguelensis morphotype, which has not been recognized before, even though the species, as well as the very material I analyzed, have been investigated intensely before by experienced diatomists; a result not disqualifying their work, but rather underlining that explicit and precise quantification of morphological information has a strong potential to generate novel scientific insights. This new morphotype has implications on the utilization of paleo-proxies which are based on geometrical valve features of F. kerguelensis. Differentiating both morphotypes might improve established methods and possibly provides a new proxy for summer sea surface temperature