174 research outputs found
Deep learning approach to Fourier ptychographic microscopy
Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems. The aim of this work is to develop a novel CNN framework to reconstruct video sequences of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided by these large spatial ensembles so as to make predictions in a sequential measurement, without using any additional temporal dataset. Specifically, we show that it is possible to reconstruct high-SBP dynamic cell videos by a CNN trained only on the first FPM dataset captured at the beginning of a time-series experiment. Our CNN approach reconstructs a 12800×10800 pixel phase image using only ∼25 seconds, a 50× speedup compared to the model-based FPM algorithm. In addition, the CNN further reduces the required number of images in each time frame by ∼ 6×. Overall, this significantly improves the imaging throughput by reducing both the acquisition and computational times. The proposed CNN is based on the conditional generative adversarial network (cGAN) framework. We further propose a mixed loss function that combines the standard image domain loss and a weighted Fourier domain loss, which leads to improved reconstruction of the high frequency information. Additionally, we also exploit transfer learning so that our pre-trained CNN can be further optimized to image other cell types. Our technique demonstrates a promising deep learning approach to continuously monitor large live-cell populations over an extended time and gather useful spatial and temporal information with sub-cellular resolution.We would like to thank NVIDIA Corporation for supporting us with the GeForce Titan Xp through the GPU Grant Program. (NVIDIA Corporation; GeForce Titan Xp through the GPU Grant Program)First author draf
Deep learning approach to Fourier ptychographic microscopy
Convolutional neural networks (CNNs) have gained tremendous success in
solving complex inverse problems. The aim of this work is to develop a novel
CNN framework to reconstruct video sequence of dynamic live cells captured
using a computational microscopy technique, Fourier ptychographic microscopy
(FPM). The unique feature of the FPM is its capability to reconstruct images
with both wide field-of-view (FOV) and high resolution, i.e. a large
space-bandwidth-product (SBP), by taking a series of low resolution intensity
images. For live cell imaging, a single FPM frame contains thousands of cell
samples with different morphological features. Our idea is to fully exploit the
statistical information provided by this large spatial ensemble so as to make
predictions in a sequential measurement, without using any additional temporal
dataset. Specifically, we show that it is possible to reconstruct high-SBP
dynamic cell videos by a CNN trained only on the first FPM dataset captured at
the beginning of a time-series experiment. Our CNN approach reconstructs a
12800X10800 pixels phase image using only ~25 seconds, a 50X speedup compared
to the model-based FPM algorithm. In addition, the CNN further reduces the
required number of images in each time frame by ~6X. Overall, this
significantly improves the imaging throughput by reducing both the acquisition
and computational times. The proposed CNN is based on the conditional
generative adversarial network (cGAN) framework. Additionally, we also exploit
transfer learning so that our pre-trained CNN can be further optimized to image
other cell types. Our technique demonstrates a promising deep learning approach
to continuously monitor large live-cell populations over an extended time and
gather useful spatial and temporal information with sub-cellular resolution
Solving ptychography with a convex relaxation
Ptychography is a powerful computational imaging technique that transforms a
collection of low-resolution images into a high-resolution sample
reconstruction. Unfortunately, algorithms that are currently used to solve this
reconstruction problem lack stability, robustness, and theoretical guarantees.
Recently, convex optimization algorithms have improved the accuracy and
reliability of several related reconstruction efforts. This paper proposes a
convex formulation of the ptychography problem. This formulation has no local
minima, it can be solved using a wide range of algorithms, it can incorporate
appropriate noise models, and it can include multiple a priori constraints. The
paper considers a specific algorithm, based on low-rank factorization, whose
runtime and memory usage are near-linear in the size of the output image.
Experiments demonstrate that this approach offers a 25% lower background
variance on average than alternating projections, the current standard
algorithm for ptychographic reconstruction.Comment: 8 pages, 8 figure
Deep learning in computational microscopy
We propose to use deep convolutional neural networks (DCNNs) to perform 2D and 3D computational imaging. Specifically, we investigate three different applications. We first try to solve the 3D inverse scattering problem based on learning a huge number of training target and speckle pairs. We also demonstrate a new DCNN architecture to perform Fourier ptychographic Microscopy (FPM) reconstruction, which achieves high-resolution phase recovery with considerably less data than standard FPM. Finally, we employ DCNN models that can predict focused 2D fluorescent microscopic images from blurred images captured at overfocused or underfocused planes.Published versio
Data preprocessing methods for robust Fourier ptychographic microscopy
Fourier ptychographic microscopy (FPM) is a recently proposed computational
imaging technique with both high resolution and wide field-of-view. In current
FP experimental setup, the dark-field images with high-angle illuminations are
easily submerged by stray light and background noise due to the low
signal-to-noise ratio, thus significantly degrading the reconstruction quality
and also imposing a major restriction on the synthetic numerical aperture (NA)
of the FP approach. To this end, an overall and systematic data preprocessing
scheme for noise removal from FP's raw dataset is provided, which involves
sampling analysis as well as underexposed/overexposed treatments, then followed
by the elimination of unknown stray light and suppression of inevitable
background noise, especially Gaussian noise and CCD dark current in our
experiments. The reported non-parametric scheme facilitates great enhancements
of the FP's performance, which has been demonstrated experimentally that the
benefits of noise removal by these methods far outweigh its defects of
concomitant signal loss. In addition, it could be flexibly cooperated with the
existing state-of-the-art algorithms, producing a stronger robustness of the FP
approach in various applications.Comment: 7 pages, 8 figure
Blind Ptychographic Phase Retrieval via Convergent Alternating Direction Method of Multipliers
Ptychography has risen as a reference X-ray imaging technique: it achieves
resolutions of one billionth of a meter, macroscopic field of view, or the
capability to retrieve chemical or magnetic contrast, among other features. A
ptychographyic reconstruction is normally formulated as a blind phase retrieval
problem, where both the image (sample) and the probe (illumination) have to be
recovered from phaseless measured data. In this article we address a nonlinear
least squares model for the blind ptychography problem with constraints on the
image and the probe by maximum likelihood estimation of the Poisson noise
model. We formulate a variant model that incorporates the information of
phaseless measurements of the probe to eliminate possible artifacts. Next, we
propose a generalized alternating direction method of multipliers designed for
the proposed nonconvex models with convergence guarantee under mild conditions,
where their subproblems can be solved by fast element-wise operations.
Numerically, the proposed algorithm outperforms state-of-the-art algorithms in
both speed and image quality.Comment: 23 page
Advanced Denoising for X-ray Ptychography
The success of ptychographic imaging experiments strongly depends on
achieving high signal-to-noise ratio. This is particularly important in
nanoscale imaging experiments when diffraction signals are very weak and the
experiments are accompanied by significant parasitic scattering (background),
outliers or correlated noise sources. It is also critical when rare events such
as cosmic rays, or bad frames caused by electronic glitches or shutter timing
malfunction take place.
In this paper, we propose a novel iterative algorithm with rigorous analysis
that exploits the direct forward model for parasitic noise and sample
smoothness to achieve a thorough characterization and removal of structured and
random noise. We present a formal description of the proposed algorithm and
prove its convergence under mild conditions. Numerical experiments from
simulations and real data (both soft and hard X-ray beamlines) demonstrate that
the proposed algorithms produce better results when compared to
state-of-the-art methods.Comment: 24 pages, 9 figure
- …