Linear chemically sensitive electron tomography using DualEELS and dictionary-based compressed sensing

We have investigated the use of DualEELS in elementally sensitive tilt series tomography in the scanning transmission electron microscope. A procedure is implemented using deconvolution to remove the effects of multiple scattering, followed by normalisation by the zero loss peak intensity. This is performed to produce a signal that is linearly dependent on the projected density of the element in each pixel. This method is compared with one that does not include deconvolution (although normalisation by the zero loss peak intensity is still performed). Additionaly, we compare the 3D reconstruction using a new compressed sensing algorithm, DLET, with the well-established SIRT algorithm. VC precipitates, which are extracted from a steel on a carbon replica, are used in this study. It is found that the use of this linear signal results in a very even density throughout the precipitates. However, when deconvolution is omitted, a slight density reduction is observed in the cores of the precipitates (a so-called cupping artefact). Additionally, it is clearly demonstrated that the 3D morphology is much better reproduced using the DLET algorithm, with very little elongation in the missing wedge direction. It is therefore concluded that reliable elementally sensitive tilt tomography using EELS requires the appropriate use of DualEELS together with a suitable reconstruction algorithm, such as the compressed sensing based reconstruction algorithm used here, to make the best use of the limited data volume and signal to noise inherent in core-loss EELS

An interactive ImageJ plugin for semi-automated image denoising in electron microscopy

The recent advent of 3D in electron microscopy (EM) has allowed for detection of nanometer resolution structures. This has caused an explosion in dataset size, necessitating the development of automated workflows. Moreover, large 3D EM datasets typically require hours to days to be acquired and accelerated imaging typically results in noisy data. Advanced denoising techniques can alleviate this, but tend to be less accessible to the community due to low-level programming environments, complex parameter tuning or a computational bottleneck. We present DenoisEM: an interactive and GPU accelerated denoising plugin for ImageJ that ensures fast parameter tuning and processing through parallel computing. Experimental results show that DenoisEM is one order of magnitude faster than related software and can accelerate data acquisition by a factor of 4 without significantly affecting data quality. Lastly, we show that image denoising benefits visualization and (semi-)automated segmentation and analysis of ultrastructure in various volume EM datasets

Dense 4D nanoscale reconstruction of living brain tissue

Three-dimensional (3D) reconstruction of living brain tissue down to an individual synapse level would create opportunities for decoding the dynamics and structure–function relationships of the brain’s complex and dense information processing network; however, this has been hindered by insufficient 3D resolution, inadequate signal-to-noise ratio and prohibitive light burden in optical imaging, whereas electron microscopy is inherently static. Here we solved these challenges by developing an integrated optical/machine-learning technology, LIONESS (live information-optimized nanoscopy enabling saturated segmentation). This leverages optical modifications to stimulated emission depletion microscopy in comprehensively, extracellularly labeled tissue and previous information on sample structure via machine learning to simultaneously achieve isotropic super-resolution, high signal-to-noise ratio and compatibility with living tissue. This allows dense deep-learning-based instance segmentation and 3D reconstruction at a synapse level, incorporating molecular, activity and morphodynamic information. LIONESS opens up avenues for studying the dynamic functional (nano-)architecture of living brain tissue

Quantitative Imaging in Electron and Confocal Microscopies for Applications in Biology

Among the large number of topics related to the quantification of images in electron and confocal microscopies for applications in biology, we selected four subjects that we consider to be representative of some recent tendencies. The first is the quantification of three-dimensional data sets recorded routinely in scanning confocal microscopy. The second is the quantification of the textural and fractal appearance of images. The two other topics are related to image series, which are more and more often provided by imaging instruments. The first kind of series concerns electron energy-filtered images. We show that the parametric (modelling) approach can be complemented by non-parametric approaches (e.g., different variants of multivariate statistical techniques). The other kind of series consists of multiple mappings of a specimen. We describe several new tools for the study and quantification of the co-location, with potential application to multiple mappings in microanalysis or in fluorescence microscopy

Efficient Bayesian-based Multi-View Deconvolution

Light sheet fluorescence microscopy is able to image large specimen with high resolution by imaging the sam- ples from multiple angles. Multi-view deconvolution can significantly improve the resolution and contrast of the images, but its application has been limited due to the large size of the datasets. Here we present a Bayesian- based derivation of multi-view deconvolution that drastically improves the convergence time and provide a fast implementation utilizing graphics hardware.Comment: 48 pages, 20 figures, 1 table, under review at Nature Method

STRUCTURED ILLUMINATION MICROSCOPE IMAGE RECONSTRUCTION USING UNROLLED PHYSICS-INFORMED GENERATIVE ADVERSARIAL NETWORK (UPIGAN)

In three-dimensional structured illumination microscopy (3D-SIM) where the images are taken from the object through the point spread function (PSF) of the imaging system, data acquisition can result in images taken under undesirable aberrations that contribute to a model mismatch. The inverse imaging problem in 3D-SIM has been solved using a variety of conventional model-based techniques that can be computationally intensive. Deep learning (DL) approaches, as opposed to traditional restoration methods, tackle the issue without access to the analytical model. This research aims to provide an unrolled physics-informed generative adversarial network (UPIGAN) for the reconstruction of 3D-SIM images utilizing data samples of mitochondria and lysosomes obtained from a 3D-SIM system. This design makes use of the benefits of physics knowledge in the unrolling step. Moreover, the GAN employs a Residual Channel Attention super-resolution deep neural network (DNN) in its generator architecture. The results indicate that the addition of both physics-informed unrolling and GAN incorporation yield improvements in reconstructed results compared to the regular DL approach