(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'

Super-resolution of 3D Magnetic Resonance Images by Random Shifting and Convolutional Neural Networks

Enhancing resolution is a permanent goal in magnetic resonance (MR) imaging, in order to keep improving diagnostic capability and registration methods. Super-resolution (SR) techniques are applied at the postprocessing stage, and their use and development have progressively increased during the last years. In particular, example-based methods have been mostly proposed in recent state-of-the-art works. In this paper, a combination of a deep-learning SR system and a random shifting technique to improve the quality of MR images is proposed, implemented and tested. The model was compared to four competitors: cubic spline interpolation, non-local means upsampling, low-rank total variation and a three-dimensional convolutional neural network trained with patches of HR brain images (SRCNN3D). The newly proposed method showed better results in Peak Signal-to-Noise Ratio, Structural Similarity index, and Bhattacharyya coefficient. Computation times were at the same level as those of these up-to-date methods. When applied to downsampled MR structural T1 images, the new method also yielded better qualitative results, both in the restored images and in the images of residuals.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Total Variation Regularized Tensor RPCA for Background Subtraction from Compressive Measurements

Background subtraction has been a fundamental and widely studied task in video analysis, with a wide range of applications in video surveillance, teleconferencing and 3D modeling. Recently, motivated by compressive imaging, background subtraction from compressive measurements (BSCM) is becoming an active research task in video surveillance. In this paper, we propose a novel tensor-based robust PCA (TenRPCA) approach for BSCM by decomposing video frames into backgrounds with spatial-temporal correlations and foregrounds with spatio-temporal continuity in a tensor framework. In this approach, we use 3D total variation (TV) to enhance the spatio-temporal continuity of foregrounds, and Tucker decomposition to model the spatio-temporal correlations of video background. Based on this idea, we design a basic tensor RPCA model over the video frames, dubbed as the holistic TenRPCA model (H-TenRPCA). To characterize the correlations among the groups of similar 3D patches of video background, we further design a patch-group-based tensor RPCA model (PG-TenRPCA) by joint tensor Tucker decompositions of 3D patch groups for modeling the video background. Efficient algorithms using alternating direction method of multipliers (ADMM) are developed to solve the proposed models. Extensive experiments on simulated and real-world videos demonstrate the superiority of the proposed approaches over the existing state-of-the-art approaches.Comment: To appear in IEEE TI

Uconnect:Synergistic Spectral CT Reconstruction With U-Nets Connecting the Energy Bins

Spectral computed tomography (CT) offers the possibility to reconstruct attenuation images at different energy levels, which can be then used for material decomposition. However, traditional methods reconstruct each energy bin individually and are vulnerable to noise. In this paper, we propose a novel synergistic method for spectral CT reconstruction, namely Uconnect. It utilizes trained convolutional neural networks (CNNs) to connect the energy bins to a latent image so that the full binned data is used synergistically. We experiment on two types of low-dose data: simulated and real patient data. Qualitative and quantitative analysis show that our proposed Uconnect outperforms state-of-art model-based iterative reconstruction (MBIR) techniques as well as CNN-based denoising

JOINT RECONSTRUCTION IN LOW DOSE MULTI-ENERGY CT

Multi-energy CT takes advantage of the non-linearly varying attenuation properties of elemental media with respect to energy, enabling more precise material identification than single-energy CT. The increased precision comes with the cost of a higher radiation dose. A straightforward way to lower the dose is to reduce the number of projections per energy, but this makes tomographic reconstruction more ill-posed. In this paper, we propose how this problem can be overcome with a combination of a regularization method that promotes structural similarity between images at different energies and a suitably selected low-dose data acquisition protocol using non-overlapping projections. The performance of various joint regularization models is assessed with both simulated and experimental data, using the novel low-dose data acquisition protocol. Three of the models are well-established, namely the joint total variation, the linear parallel level sets and the spectral smoothness promoting regularization models. Furthermore, one new joint regularization model is introduced for multi-energy CT: a regularization based on the structure function from the structural similarity index. The findings show that joint regularization outperforms individual channel-by-channel reconstruction. Furthermore, the proposed combination of joint reconstruction and non-overlapping projection geometry enables significant reduction of radiation dose.Peer reviewe