Lose The Views: Limited Angle CT Reconstruction via Implicit Sinogram Completion

Computed Tomography (CT) reconstruction is a fundamental component to a wide variety of applications ranging from security, to healthcare. The classical techniques require measuring projections, called sinograms, from a full 180$^\circ$ view of the object. This is impractical in a limited angle scenario, when the viewing angle is less than 180$^\circ$, which can occur due to different factors including restrictions on scanning time, limited flexibility of scanner rotation, etc. The sinograms obtained as a result, cause existing techniques to produce highly artifact-laden reconstructions. In this paper, we propose to address this problem through implicit sinogram completion, on a challenging real world dataset containing scans of common checked-in luggage. We propose a system, consisting of 1D and 2D convolutional neural networks, that operates on a limited angle sinogram to directly produce the best estimate of a reconstruction. Next, we use the x-ray transform on this reconstruction to obtain a "completed" sinogram, as if it came from a full 180$^\circ$ measurement. We feed this to standard analytical and iterative reconstruction techniques to obtain the final reconstruction. We show with extensive experimentation that this combined strategy outperforms many competitive baselines. We also propose a measure of confidence for the reconstruction that enables a practitioner to gauge the reliability of a prediction made by our network. We show that this measure is a strong indicator of quality as measured by the PSNR, while not requiring ground truth at test time. Finally, using a segmentation experiment, we show that our reconstruction preserves the 3D structure of objects effectively.Comment: Spotlight presentation at CVPR 201

The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography

We present the ASTRA Toolbox as an open platform for 3D image reconstruction in tomography. Most of the software tools that are currently used in electron tomography offer limited flexibility with respect to the geometrical parameters of the acquisition model and the algorithms used for reconstruction. The ASTRA Toolbox provides an extensive set of fast and flexible building blocks that can be used to develop advanced reconstruction algorithms, effectively removing these limitations. We demonstrate this flexibility, the resulting reconstruction quality, and the computational efficiency of this toolbox by a series of experiments, based on experimental dual-axis tilt series

Progress and opportunities in EELS and EDS tomography.

Electron tomography using energy loss and X-ray spectroscopy in the electron microscope continues to develop in rapidly evolving and diverse directions, enabling new insight into the three-dimensional chemistry and physics of nanoscale volumes. Progress has been made recently in improving reconstructions from EELS and EDS signals in electron tomography by applying compressed sensing methods, characterizing new detector technologies in detail, deriving improved models of signal generation, and exploring machine learning approaches to signal processing. These disparate threads can be brought together in a cohesive framework in terms of a model-based approach to analytical electron tomography. Models incorporate information on signal generation and detection as well as prior knowledge of structures in the spectrum image data. Many recent examples illustrate the flexibility of this approach and its feasibility for addressing challenges in non-linear or limited signals in EELS and EDS tomography. Further work in combining multiple imaging and spectroscopy modalities, developing synergistic data acquisition, processing, and reconstruction approaches, and improving the precision of quantitative spectroscopic tomography will expand the frontiers of spatial resolution, dose limits, and maximal information recovery.SMC acknowledges support from the EPSRC Cambridge NanoDTC, EP/G037221/1, and Trinity College, Cambridge. SMC and PAM also acknowledge support from the European Research Council under the European Union's Seventh Framework Program (No. FP7/2007–2013)/ERC Grant Agreement No. 291522-3DIMAGE

CHARACTERIZATION OF HETEROGENEOUS CATALYSTS USING ADVANCED TRANSMISSION ELECTRON MICROSCOPY TECHNIQUES

Chang Wan Han, Ph. D., Purdue University, December 2016. Characterization of Heterogeneous Catalysts using Advanced Transmission Electron Microscopy Techniques. Major Professor: Volkan Ortalan

Complementary 2D/3D Imaging of Functional Materials using X-ray & Electron Microscopy

Catalysts and other functional materials are generally hierarchically structured materials. Hence, the detailed characterization at different length scales, and especially under reaction conditions, are necessary to unravel their mechanisms and to improve their performance and catalytic activities. Besides, a combination of several techniques is required to acquire complementary information owing to the lack of a single technique able to cover all the length scales. With respect to length, the best way to investigate is by microscopy either in 2D or more preferably in 3D. The work began with an exploration of three different 3D imaging techniques, i.e. ptychographic X-ray computed tomography, electron tomography, and focused ion beam slice-and view. Using nanoporous gold as the model, this study aimed to exhibit the versatility of 3D microscopy as a method beyond imaging as well as to confirm the necessity of complementary nature between them, where electron offers better spatial resolution and X-ray provides larger field of view. The study then continued by utilizing ptychographic X-ray computed tomography for quasi in situ thermal treatment of the same materials under atmospheric pressure. This study highlighted its ease of use of implementing in situ studies, complemented by electron tomography to prove its powerful ability to resolve what ptychographic tomography cannot. The resulting 3D volumes were then used for air permeability and CO2 diffusion simulations, along with material’s electrical and thermal conductivity simulations in order to further expose another excellent benefit from 3D microscopy. Ultimately, the work proceeded into developing two cells in order to perform full in situ investigations under controlled temperatures and atmospheres, where one cell was built for 2D only (X-ray) ptychography experiments with simultaneous X-ray fluorescence mapping, and the other was constructed with an additional capability for 3D limited-angle ptychographic tomography experiments. The feasibility tests were conducted using several functional materials, i.e. nanoporous gold, zeolite, and cobalt-manganese-oxides hollow sphere, as the models for thermal annealing process under specific atmospheres. This work eventually attests the importance of in situ studies in precisely determining the onset annealing temperatures under particular environments, to visualize the morphology online either in 2D or 3D, and to simultaneously map elemental distributions live. Moreover, a complementary technique via transmission electron microscopy was also demonstrated on the same sample, adding up another advantage in using the cells. Despite the preliminary results from in situ limited-angle ptychographic tomography experiments for limitation in data reconstruction, a new tomographic reconstruction technique was recently developed as a solution to acquire 3D images with shortened acquisition times. In conclusions, the work here converges into the ideal case of performing all-around in situ 3D imaging of functional materials for quantitative analysis and simulation

Statistical Reconstruction Methods for 3D Imaging of Biological Samples with Electron Microscopy

Electron microscopy has emerged as the leading method for the in vivo study of biological structures such as cells, organelles, protein molecules and virus like particles. By providing 3D images up to near atomic resolution, it plays a significant role in analyzing complex organizations, understanding physiological functions and developing medicines. The 3D images representing the electrostatic potential distribution are reconstructed by utilizing the 2D projection images of the target acquired by electron microscope. There are two main 3D reconstruction techniques in the field of electron microscopy: electron tomography (ET) and single particle reconstruction (SPR). In ET, the projection images are acquired by rotating the specimen for different angles. In SPR, the projection images are obtained by analyzing the images of multiple objects representing the same structure. Then, the tomographic reconstruction methods are applied in both methods to obtain the 3D image through the 2D projections.Physical and mechanical limitations can prevent to acquire projection images that cover the projection angle space completely and uniformly. Incomplete and non-uniform sampling of the projection angles results in anisotropic resolution in the image plane and generates artifacts. Another problem is that the total applied dose of electrons is limited in order to prevent the radiation damage to the biological target. Therefore, limited number of projection images with low signal to noise ratio can be used in the reconstruction process. This affects the resolution of the reconstructed image significantly. This study presents statistical methods to overcome these major challenges to obtain precise and high resolution images in electron microscopy.Statistical image reconstruction methods have been successful in recovering a signal from imperfect measurements due to their capability of utilizing a priori information. First, we developed a sequential application of a statistical method for ET. Then we extended the method to support projection angles freely distributed in 3D space and applied the method in SPR. In both applications, we observed the strength of the method in projection gap filling, robustness against noise, and resolving the high resolution details in comparison with the conventional reconstruction methods. Afterwards, we improved the method in terms of computation time by incorporating multiresolution reconstruction. Furthermore, we developed an adaptive regularization method to minimize the parameters required to be set by the user. We also proposed the local adaptive Wiener filter for the class averaging step of SPR to improve the averaging accuracy.The qualitative and quantitative analysis of the reconstructions with phantom and experimental datasets has demonstrated that the proposed reconstruction methods outperform the conventional reconstruction methods. These statistical approaches provided better image accuracy and higher resolution compared with the conventional algebraic and transfer domain based reconstruction methods. The methods provided in this study contribute to enhance our understanding of cellular and molecular structures by providing 3D images of those with improved accuracy and resolution