DataSheet_1_Deep focus-extended darkfield imaging for in situ observation of marine plankton.pdf

Darkfield imaging can achieve in situ observation of marine plankton with unique advantages of high-resolution, high-contrast and colorful imaging for plankton species identification, size measurement and abundance estimation. However, existing underwater darkfield imagers have very shallow depth-of-field, leading to inefficient seawater sampling for plankton observation. We develop a data-driven method that can algorithmically refocus planktonic objects in their defocused darkfield images, equivalently achieving focus-extension for their acquisition imagers. We devise a set of dual-channel imaging apparatus to quickly capture paired images of live plankton with different defocus degrees in seawater samples, simulating the settings as in in situ darkfield plankton imaging. Through a series of registration and preprocessing operations on the raw image pairs, a dataset consisting of 55 000 pairs of defocused-focused plankter images have been constructed with an accurate defocus distance label for each defocused image. We use the dataset to train an end-to-end deep convolution neural network named IsPlanktonFE, and testify its focus-extension performance through extensive experiments. The experimental results show that IsPlanktonFE has extended the depth-of-field of a 0.5× darkfield imaging system to ~7 times of its original value. Moreover, the model has exhibited good content and instrument generalizability, and considerable accuracy improvement for a pre-trained ResNet-18 network to classify defocused plankton images. This focus-extension technology is expected to greatly enhance the sampling throughput and efficiency for the future in situ marine plankton observation systems, and promote the wide applications of darkfield plankton imaging instruments in marine ecology research and aquatic environment monitoring programs.</p

Simulated phantoms used in algorithm validation, with theoretical fractal dimensions ranging between 1 and 3.

(a) Circle: radius = 8, image size: 120 x 120, line width is 1 pixel (theoretical FD = 1). (b) Fourth-iteration Koch, image size: 283 x 84, line width is 1 pixel (theoretical FD = 1.2619). (c) 3D random Cantor set with p = 0.7, image size: 128 x 128 x 128, Voxels set to 1 (theoretical FD = 2.485).</p

Differences in fractal dimension for left hippocampus, right hippocampus and left thalamus between schizophrenia patients and healthy controls.

<p>Each data point represents <i>D</i>₁ information dimension value for each participant for (a) Left hippocampus, (b) Right hippocampus, and (c) Left thalamus. The black dash-dot line and the magenta dash-dash line denote median fractal dimension values, for schizophrenia patients and healthy control groups, respectively. Significantly lower FD values were found for schizophrenia patients relative to healthy controls (Mann-Whitney <i>U</i> test, <i>p</i>< 0.05; FDR correction). <i>Note</i>. SCZ: patients with schizophrenia; HC: healthy controls.</p

Fractal dimension values for subcortical structures.

<p>Fractal dimension values for subcortical structures.</p

Computing fractal dimension using 3D information measure.

<p>Information dimension, <i>D</i>₁, measure. In the scatterplot of log(1/<i>r</i>) versus log(<i>I</i>(<i>r</i>)), <i>r</i> is box size and <i>I</i>(<i>r</i>) is the information theoretic entropy for the box size <i>r</i>. <i>Note</i>. For information measure, the initial, pre-determined range of box sizes is <i>r</i> = 2… 30 voxels (in increments of 1 voxel). Data shown are for left hippocampus from one healthy control participant. Linear regression analysis is performed iteratively. Blue line indicates the linear fit over the entire range of <i>r</i>. Red dotted line indicates the final fit (<i>R</i>²); the slope of this line corresponds to the fractal dimension, <i>D</i>₁. Breakpoint separates non-linear data points from the data used in the final regression analysis. ln denotes natural log. <i>Min r</i> is the new smallest box size and <i>Max r</i> is the new largest box size.</p

Illustration of fractal self-similarity.

<p>(a) A Sierpinski triangle is an example of a pure fractal. A small portion of the triangle looks exactly like the whole triangle. (b) Self-similarity holds across a limited range of spatial scales for a natural object such as this Romanesco Broccoli (Photos courtesy of Live Earth Farm).</p

Fractal dimension values and box size range of phantoms.

<p>Fractal dimension values and box size range of phantoms.</p

Illustration of subtle surface non-linearities in schizophrenia as captured by the FD measure, using individual participants’ data for left and right hippocampus.

<p>(a) Representative healthy control, and (b) Patient with schizophrenia. Left panel shows left hippocampus, right panel shows right hippocampus. In (b), shadow highlight indicates data points used in the final fit. Subtle deviations from linearity are seen in (b), which shows data points whose <i>I</i>(r) counts deviate relative to the line of best fit. In the left panel, an example of this can be observed at a point with x and y coordinates [-1.946, 2.652] (fourth from the bottom of shaded area) and at [-1.386, 3.608] (third from the top), and in the right panel, at [-2.079, 2.445] (third from the bottom) (please see main text for detailed explanation). Insets show reconstructions of left and right hippocampi for these participants; 1 cube represents 1 voxel (1.5 x 1.5 x 1.5 mm).</p