Solving ill-posed inverse problems using iterative deep neural networks

We propose a partially learned approach for the solution of ill posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularization theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularizing functional. The method results in a gradient-like iterative scheme, where the "gradient" component is learned using a convolutional network that includes the gradients of the data discrepancy and regularizer as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against FBP and TV reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the TV reconstruction while being significantly faster, giving reconstructions of 512 x 512 volumes in about 0.4 seconds using a single GPU

Lose The Views: Limited Angle CT Reconstruction via Implicit Sinogram Completion

Computed Tomography (CT) reconstruction is a fundamental component to a wide variety of applications ranging from security, to healthcare. The classical techniques require measuring projections, called sinograms, from a full 180$^\circ$ view of the object. This is impractical in a limited angle scenario, when the viewing angle is less than 180$^\circ$, which can occur due to different factors including restrictions on scanning time, limited flexibility of scanner rotation, etc. The sinograms obtained as a result, cause existing techniques to produce highly artifact-laden reconstructions. In this paper, we propose to address this problem through implicit sinogram completion, on a challenging real world dataset containing scans of common checked-in luggage. We propose a system, consisting of 1D and 2D convolutional neural networks, that operates on a limited angle sinogram to directly produce the best estimate of a reconstruction. Next, we use the x-ray transform on this reconstruction to obtain a "completed" sinogram, as if it came from a full 180$^\circ$ measurement. We feed this to standard analytical and iterative reconstruction techniques to obtain the final reconstruction. We show with extensive experimentation that this combined strategy outperforms many competitive baselines. We also propose a measure of confidence for the reconstruction that enables a practitioner to gauge the reliability of a prediction made by our network. We show that this measure is a strong indicator of quality as measured by the PSNR, while not requiring ground truth at test time. Finally, using a segmentation experiment, we show that our reconstruction preserves the 3D structure of objects effectively.Comment: Spotlight presentation at CVPR 201

MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, MeshfreeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Partial Differential Equation (PDE) constraints to be imposed, and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal domains owing to its fully convolutional encoder. We empirically study the performance of MeshfreeFlowNet on the task of super-resolution of turbulent flows in the Rayleigh-Benard convection problem. Across a diverse set of evaluation metrics, we show that MeshfreeFlowNet significantly outperforms existing baselines. Furthermore, we provide a large scale implementation of MeshfreeFlowNet and show that it efficiently scales across large clusters, achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of less than 4 minutes.Comment: Supplementary Video: https://youtu.be/mjqwPch9gDo. Accepted to SC2

Electrical Impedance Tomography (EIT): The Establishment of a Dual Current Stimulation EIT System for Improved Image Quality

Electrical Impedance Tomography (EIT) is a noninvasive imaging technique that reproduces images of cross-sections, based on the internal impedance distribution of an object. This Dissertation investigates and confirms the use of a dual current stimulation EIT (DCS EIT) system. The results of this investigation presented a size error of 2.82 % and a position error of 5.93 % in the reconstructed images, when compared to the actual size and position of the anomaly inside a test object. These results confirmed that the DCS EIT system produced images of superior quality (fewer image reconstruction errors) to those produced from reviewed single plane stimulating EIT systems, which confirmed the research hypothesis. This system incorporates two independent current stimulating patterns, which establishes a more even distribution of current in the test object, compared to single plane systems, and is more efficient than 2.5D EIT systems because the DCS EIT system only measures boundary voltages in the center plane, compared to 2.5D EIT systems that measure the boundary voltages in all electrode planes. The system uses 48 compound electrodes, divided into three electrode planes. Current is sourced and sunk perpendicularly in the center plane, to produce a high current density near the center of the test object. Sequentially, current is sourced through an electrode in the top electrode plane and sunk through an electrode in the bottom plane, directly below the source electrode, to produce a high current density near the boundary of the test object, in the center plane. During both injection cycles, boundary potentials are measured in the center plane. Following the measurement of a complete frame, a weighted average is computed from the single and cross plane measured data. The weighted measured voltages, injected currents and Finite Element Model of the object is used to reconstruct an image of the internal impedance distribution along a cross-section of the object. This method is applicable to the biomedical imaging and process monitoring fields

Recent developments in X-ray diffraction/scattering computed tomography for materials science

X-ray diffraction/scattering computed tomography (XDS-CT) methods are a non-destructive class of chemical imaging techniques that have the capacity to provide reconstructions of sample cross-sections with spatially resolved chemical information. While X-ray diffraction CT (XRD-CT) is the most well-established method, recent advances in instrumentation and data reconstruction have seen greater use of related techniques like small angle X-ray scattering CT and pair distribution function CT. Additionally, the adoption of machine learning techniques for tomographic reconstruction and data analysis are fundamentally disrupting how XDS-CT data is processed. The following narrative review highlights recent developments and applications of XDS-CT with a focus on studies in the last five years. This article is part of the theme issue 'Exploring the length scales, timescales and chemistry of challenging materials (Part 2)'

Recent developments in X-ray diffraction/scattering computed tomography for materials science

X-ray diffraction/scattering computed tomography (XDS-CT) methods are a non-destructive class of chemical imaging techniques that have the capacity to provide reconstructions of sample cross-sections with spatially resolved chemical information. While X-ray diffraction CT (XRD-CT) is the most well-established method, recent advances in instrumentation and data reconstruction have seen greater use of related techniques like small angle X-ray scattering CT and pair distribution function CT. Additionally, the adoption of machine learning techniques for tomographic reconstruction and data analysis are fundamentally disrupting how XDS-CT data is processed. The following narrative review highlights recent developments and applications of XDS-CT with a focus on studies in the last five years. This article is part of the theme issue 'Exploring the length scales, timescales and chemistry of challenging materials (Part 2)'

NeBLa: Neural Beer-Lambert for 3D Reconstruction of Oral Structures from Panoramic Radiographs

Panoramic radiography (panoramic X-ray, PX) is a widely used imaging modality for dental examination. However, its applicability is limited as compared to 3D Cone-beam computed tomography (CBCT), because PX only provides 2D flattened images of the oral structure. In this paper, we propose a new framework which estimates 3D oral structure from real-world PX images. Since there are not many matching PX and CBCT data, we used simulated PX from CBCT for training, however, we used real-world panoramic radiographs at the inference time. We propose a new ray-sampling method to make simulated panoramic radiographs inspired by the principle of panoramic radiography along with the rendering function derived from the Beer-Lambert law. Our model consists of three parts: translation module, generation module, and refinement module. The translation module changes the real-world panoramic radiograph to the simulated training image style. The generation module makes the 3D structure from the input image without any prior information such as a dental arch. Our ray-based generation approach makes it possible to reverse the process of generating PX from oral structure in order to reconstruct CBCT data. Lastly, the refinement module enhances the quality of the 3D output. Results show that our approach works better for simulated and real-world images compared to other state-of-the-art methods.Comment: 10 pages, 4 figure