Fully 3D Implementation of the End-to-end Deep Image Prior-based PET Image Reconstruction Using Block Iterative Algorithm

Deep image prior (DIP) has recently attracted attention owing to its unsupervised positron emission tomography (PET) image reconstruction, which does not require any prior training dataset. In this paper, we present the first attempt to implement an end-to-end DIP-based fully 3D PET image reconstruction method that incorporates a forward-projection model into a loss function. To implement a practical fully 3D PET image reconstruction, which could not be performed due to a graphics processing unit memory limitation, we modify the DIP optimization to block-iteration and sequentially learn an ordered sequence of block sinograms. Furthermore, the relative difference penalty (RDP) term was added to the loss function to enhance the quantitative PET image accuracy. We evaluated our proposed method using Monte Carlo simulation with [$^{18}$F]FDG PET data of a human brain and a preclinical study on monkey brain [$^{18}$F]FDG PET data. The proposed method was compared with the maximum-likelihood expectation maximization (EM), maximum-a-posterior EM with RDP, and hybrid DIP-based PET reconstruction methods. The simulation results showed that the proposed method improved the PET image quality by reducing statistical noise and preserved a contrast of brain structures and inserted tumor compared with other algorithms. In the preclinical experiment, finer structures and better contrast recovery were obtained by the proposed method. This indicated that the proposed method can produce high-quality images without a prior training dataset. Thus, the proposed method is a key enabling technology for the straightforward and practical implementation of end-to-end DIP-based fully 3D PET image reconstruction.Comment: 9 pages, 10 figure

Potentials and caveats of AI in Hybrid Imaging

State-of-the-art patient management frequently mandates the investigation of both anatomy and physiology of the patients. Hybrid imaging modalities such as the PET/MRI, PET/CT and SPECT/CT have the ability to provide both structural and functional information of the investigated tissues in a single examination. With the introduction of such advanced hardware fusion, new problems arise such as the exceedingly large amount of multi-modality data that requires novel approaches of how to extract a maximum of clinical information from large sets of multi-dimensional imaging data. Artificial intelligence (AI) has emerged as one of the leading technologies that has shown promise in facilitating highly integrative analysis of multi-parametric data. Specifically, the usefulness of AI algorithms in the medical imaging field has been heavily investigated in the realms of (1) image acquisition and reconstruction, (2) post-processing and (3) data mining and modelling. Here, we aim to provide an overview of the challenges encountered in hybrid imaging and discuss how AI algorithms can facilitate potential solutions. In addition, we highlight the pitfalls and challenges in using advanced AI algorithms in the context of hybrid imaging and provide suggestions for building robust AI solutions that enable reproducible and transparent research

Deep learning assisted MRI guided attenuation correction in PET

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonPositron emission tomography (PET) is a unique imaging modality that provides physiological and functional details of the tissue at the molecular level. However, the acquired PET images have some limitations such as the attenuation. PET attenuation correction is an essential step to obtain the full potential of PET quantification. With the wide use of hybrid PET/MR scanners, magnetic resonance (MR) images are used to address the problem of PET attenuation correction. The MR images segmentation is one simple and robust approach to create pseudo computed tomography (CT) images, which are used to generate attenuation coefficient maps to correct the PET attenuation. Recently, deep learning has been proposed and used as a promising technique to efficiently perform MR and various medical images segmentation. In this research work, deep learning guided segmentation approaches have been proposed to enhance the bone class segmentation of MR brain images in order to generate accurate pseudo-CT images. The first approach has introduced the combination of handcrafted features with deep learning features to enrich the set of features. Multiresolution analysis techniques, which generate multiscale and multidirectional coefficients of an image such as contourlet and shearlet transforms, are applied and combined with deep convolutional neural network (CNN) features. Different experiments have been conducted to investigate the number of selected coefficients and the insertion location of the handcrafted features. The second approach aims at reducing the segmentation algorithm’s complexity while maintaining the segmentation performance. An attention based convolutional encode-decoder network has been proposed to adaptively recalibrate the deep network features. This attention based network consists of two different squeeze and excitation blocks that excite the features spatially and channel wise. The two blocks are combined sequentially to decrease the number of network’s parameters and reduces the model complexity. The third approach has been focuses on the application of transfer learning from different MR sequences such as T1 weighted (T1-w) and T2 weighted (T2-w) images. A pretrained model with T1-w MR sequences is fine tuned to perform the segmentation of T2-w images. Multiple fine tuning approaches and experiments have been conducted to study the best fine tuning mechanism that is able to build an efficient segmentation model for both T1-w and T2-w segmentation. Clinical datasets of fifty patients with different conditions and diagnosis have been used to carry an objective evaluation to measure the segmentation performance of the results obtained by the three proposed methods. The first and second approaches have been validated with other studies in the literature that applied deep network based segmentation technique to perform MR based attenuation correction for PET images. The proposed methods have shown an enhancement in the bone segmentation with an increase of dice similarity coefficient (DSC) from 0.6179 to 0.6567 using an ensemble of CNNs with an improvement percentage of 6.3%. The proposed excitation-based CNN has decreased the model complexity by decreasing the number of trainable parameters by more than 46% where less computing resources are required to train the model. The proposed hybrid transfer learning method has shown its superiority to build a multi-sequences (T1-w and T2-w) segmentation approach compared to other applied transfer learning methods especially with the bone class where the DSC is increased from 0.3841 to 0.5393. Moreover, the hybrid transfer learning approach requires less computing time than transfer learning using open and conservative fine tuning

PET/MRI 및 MR-IGRT를 위한 MRI 기반 합성 CT 생성의 타당성 연구

학위논문 (박사) -- 서울대학교 대학원 : 의과대학 의과학과, 2020. 8. 이재성.Over the past decade, the application of magnetic resonance imaging (MRI) in the field of diagnosis and treatment has increased. MRI provides higher soft-tissue contrast, especially in the brain, abdominal organ, and bone marrow without the expose of ionizing radiation. Hence, simultaneous positron emission tomography/MR (PET/MR) system and MR-image guided radiation therapy (MR-IGRT) system has recently been emerged and currently available for clinical study. One major issue in PET/MR system is attenuation correction from MRI scans for PET quantification and a similar need for the assignment of electron densities to MRI scans for dose calculation can be found in MR-IGRT system. Because the MR signals are related to the proton density and relaxation properties of tissue, not to electron density. To overcome this problem, the method called synthetic CT (sCT), a pseudo CT derived from MR images, has been proposed. In this thesis, studies on generating synthetic CT and investigating the feasibility of using a MR-based synthetic CT for diagnostic and radiotherapy application were presented. Firstly, MR image-based attenuation correction (MR-AC) method using level-set segmentation for brain PET/MRI was developed. To resolve conventional inaccuracy MR-AC problem, we proposed an improved ultrashort echo time MR-AC method that was based on a multiphase level-set algorithm with main magnetic field inhomogeneity correction. We also assessed the feasibility of level-set based MR-AC method, compared with CT-AC and MR-AC provided by the manufacturer of the PET/MRI scanner. Secondly, we proposed sCT generation from the low field MR images using 2D convolution neural network model for MR-IGRT system. This sCT images were compared to the deformed CT generated using the deformable registration being used in the current system. We assessed the feasibility of using sCT for radiation treatment planning from each of the patients with pelvic, thoraic and abdominal region through geometric and dosimetric evaluation.지난 10년간 진단 및 치료분야에서 자기공명영상(Magnetic resonance imaging; MRI) 의 적용이 증가하였다. MRI는 CT와 비교해 추가적인 전리방사선의 피폭없이 뇌, 복부 기관 및 골수 등에서 더 높은 연조직 대비를 제공한다. 따라서 MRI를 적용한 양전자방출단층촬영(Positron emission tomography; PET)/MR 시스템과 MR 영상 유도 방사선 치료 시스템(MR-image guided radiation therapy; MR-IGRT)이 진단 및 치료 방사선분야에 등장하여 임상에 사용되고 있다. PET/MR 시스템의 한 가지 주요 문제는 PET 정량화를 위한 MRI 스캔으로부터의 감쇠 보정이며, MR-IGRT 시스템에서 선량 계산을 위해 MR 영상에 전자 밀도를 할당하는 것과 비슷한 필요성을 찾을 수 있다. 이는 MR 신호가 전자 밀도가 아닌 조직의 양성자 밀도 및 T1, T2 이완 특성과 관련이 있기 때문이다. 이 문제를 극복하기 위해, MR 이미지로부터 유래된 가상의 CT인 합성 CT라 불리는 방법이 제안되었다. 본 학위논문에서는 합성 CT 생성 방법 및 진단 및 방사선 치료에 적용을 위한 MR 영상 기반 합성 CT 사용의 임상적 타당성을 조사하였다. 첫째로, 뇌 PET/MR를 위한 레벨셋 분할을 이용한 MR 이미지 기반 감쇠 보정 방법을 개발하였다. MR 이미지 기반 감쇠 보정의 부정확성은 정량화 오류와 뇌 PET/MRI 연구에서 병변의 잘못된 판독으로 이어진다. 이 문제를 해결하기 위해, 자기장 불균일 보정을 포함한 다상 레벨셋 알고리즘에 기초한 개선된 초단파 에코 시간 MR-AC 방법을 제안하였다. 또한 CT-AC 및 PET/MRI 스캐너 제조업체가 제공한 MR-AC와 비교하여 레벨셋 기반 MR-AC 방법의 임상적 사용가능성을 평가하였다. 둘째로, MR-IGRT 시스템을 위한 심층 컨볼루션 신경망 모델을 사용하여 저필드 MR 이미지에서 생성된 합성 CT 방법를 제안하였다. 이 합성 CT 이미지를 변형 정합을 사용하여 생성된 변형 CT와 비교 하였다. 또한 골반, 흉부 및 복부 환자에서의 기하학적, 선량적 분석을 통해 방사선 치료계획에서의 합성 CT를 사용가능성을 평가하였다.Chapter 1. Introduction 1 1.1. Background 1 1.1.1. The Integration of MRI into Other Medical Devices 1 1.1.2. Chanllenges in the MRI Integrated System 4 1.1.3. Synthetic CT Generation 5 1.2. Purpose of Research 6 Chapter 2. MRI-based Attenuation Correction for PET/MRI 8 2.1. Background 8 2.2. Materials and Methods 10 2.2.1. Brain PET Dataset 19 2.2.2. MR-Based Attenuation Map using Level-Set Algorithm 12 2.2.3. Image Processing and Reconstruction 18 2.3. Results 20 2.4. Discussion 28 Chapter 3. MRI-based synthetic CT generation for MR-IGRT 30 3.1. Background 30 3.2. Materials and Methods 32 3.2.1. MR-dCT Paired DataSet 32 3.2.2. Synthetic CT Generation using 2D CNN 36 3.2.3. Data Analysis 38 3.3. Results 41 3.3.1. Image Comparison 41 3.3.2. Geometric Analysis 49 3.3.3. Dosimetric Analysis 49 3.4. Discussion 56 Chapter 4. Conclusions 59 Bibliography 60 Abstract in Korean (국문 초록) 64Docto

Quantitative PET and SPECT

Since the introduction of personalized medicine, the primary focus of imaging has moved from detection and diagnosis to tissue characterization, the determination of prognosis, prediction of treatment efficacy, and measurement of treatment response. Precision (personalized) imaging heavily relies on the use of hybrid technologies and quantitative imaging biomarkers. The growing number of promising theragnostics require accurate quantification for pre- and post-treatment dosimetry. Furthermore, quantification is required in the pharmacokinetic analysis of new tracers and drugs and in the assessment of drug resistance. Positron Emission Tomography (PET) is, by nature, a quantitative imaging tool, relating the time–activity concentration in tissues and the basic functional parameters governing the biological processes being studied. Recent innovations in single photon emission computed tomography (SPECT) reconstruction techniques have allowed for SPECT to move from relative/semi-quantitative measures to absolute quantification. The strength of PET and SPECT is that they permit whole-body molecular imaging in a noninvasive way, evaluating multiple disease sites. Furthermore, serial scanning can be performed, allowing for the measurement of functional changes over time during therapeutic interventions. This Special Issue highlights the hot topics on quantitative PET and SPECT

Deep MR to CT Synthesis for PET/MR Attenuation Correction

Positron Emission Tomography - Magnetic Resonance (PET/MR) imaging combines the functional information from PET with the flexibility of MR imaging. It is essential, however, to correct for photon attenuation when reconstructing PETs, which is challenging for PET/MR as neither modality directly image tissue attenuation properties. Classical MR-based computed tomography (CT) synthesis methods, such as multi-atlas propagation, have been the method of choice for PET attenuation correction (AC), however, these methods are slow and suffer from the poor ability to handle anatomical abnormalities. To overcome this limitation, this thesis explores the rising field of artificial intelligence in order to develop novel methods for PET/MR AC. Deep learning-based synthesis methods such as the standard U-Net architecture are not very stable, accurate, and robust to small variations in image appearance. Thus, the first proposed MR to CT synthesis method deploys a boosting strategy, where multiple weak predictors build a strong predictor providing a significant improvement in CT and PET reconstruction accuracy. Standard deep learning-based methods as well as more advanced methods like the first proposed method show issues in the presence of very complex imaging environments and large images such as whole-body images. The second proposed method learns the image context between whole-body MRs and CTs through multiple resolutions while simultaneously modelling uncertainty. Lastly, as the purpose of synthesizing a CT is to better reconstruct PET data, the use of CT-based loss functions is questioned within this thesis. Such losses fail to recognize the main objective of MR-based AC, which is to generate a synthetic CT that, when used for PET AC, makes the reconstructed PET as close as possible to the gold standard PET. The third proposed method introduces a novel PET-based loss that minimizes CT residuals with respect to the PET reconstruction

[¹⁸F]fluorination of biorelevant arylboronic acid pinacol ester scaffolds synthesized by convergence techniques

Aim: The development of small molecules through convergent multicomponent reactions (MCR) has been boosted during the last decade due to the ability to synthesize, virtually without any side-products, numerous small drug-like molecules with several degrees of structural diversity.(1) The association of positron emission tomography (PET) labeling techniques in line with the “one-pot” development of biologically active compounds has the potential to become relevant not only for the evaluation and characterization of those MCR products through molecular imaging, but also to increase the library of radiotracers available. Therefore, since the [18F]fluorination of arylboronic acid pinacol ester derivatives tolerates electron-poor and electro-rich arenes and various functional groups,(2) the main goal of this research work was to achieve the 18F-radiolabeling of several different molecules synthesized through MCR. Materials and Methods: [18F]Fluorination of boronic acid pinacol esters was first extensively optimized using a benzaldehyde derivative in relation to the ideal amount of Cu(II) catalyst and precursor to be used, as well as the reaction solvent. Radiochemical conversion (RCC) yields were assessed by TLC-SG. The optimized radiolabeling conditions were subsequently applied to several structurally different MCR scaffolds comprising biologically relevant pharmacophores (e.g. β-lactam, morpholine, tetrazole, oxazole) that were synthesized to specifically contain a boronic acid pinacol ester group. Results: Radiolabeling with fluorine-18 was achieved with volumes (800 μl) and activities (≤ 2 GBq) compatible with most radiochemistry techniques and modules. In summary, an increase in the quantities of precursor or Cu(II) catalyst lead to higher conversion yields. An optimal amount of precursor (0.06 mmol) and Cu(OTf)2(py)4 (0.04 mmol) was defined for further reactions, with DMA being a preferential solvent over DMF. RCC yields from 15% to 76%, depending on the scaffold, were reproducibly achieved. Interestingly, it was noticed that the structure of the scaffolds, beyond the arylboronic acid, exerts some influence in the final RCC, with electron-withdrawing groups in the para position apparently enhancing the radiolabeling yield. Conclusion: The developed method with high RCC and reproducibility has the potential to be applied in line with MCR and also has a possibility to be incorporated in a later stage of this convergent “one-pot” synthesis strategy. Further studies are currently ongoing to apply this radiolabeling concept to fluorine-containing approved drugs whose boronic acid pinacol ester precursors can be synthesized through MCR (e.g. atorvastatin)

The role of deep learning in structural and functional lung imaging

Background: Structural and functional lung imaging are critical components of pulmonary patient care. Image analysis methods, such as image segmentation, applied to structural and functional lung images, have significant benefits for patients with lung pathologies, including the computation of clinical biomarkers. Traditionally, machine learning (ML) approaches, such as clustering, and computational modelling techniques, such as CT-ventilation imaging, have been used for segmentation and synthesis, respectively. Deep learning (DL) has shown promise in medical image analysis tasks, often outperforming alternative methods. Purpose: To address the hypothesis that DL can outperform conventional ML and classical image analysis methods for the segmentation and synthesis of structural and functional lung imaging via: i. development and comparison of 3D convolutional neural networks (CNNs) for the segmentation of ventilated lung using hyperpolarised (HP) gas MRI. ii. development of a generalisable, multi-centre CNN for segmentation of the lung cavity using 1H-MRI. iii. the proposal of a framework for estimating the lung cavity in the spatial domain of HP gas MRI. iv. development of a workflow to synthesise HP gas MRI from multi-inflation, non-contrast CT. v. the proposal of a framework for the synthesis of fully-volumetric HP gas MRI ventilation from a large, diverse dataset of non-contrast, multi-inflation 1H-MRI scans. Methods: i. A 3D CNN-based method for the segmentation of ventilated lung using HP gas MRI was developed and CNN parameters, such as architecture, loss function and pre-processing were optimised. ii. A 3D CNN trained on a multi-acquisition dataset and validated on data from external centres was compared with a 2D alternative for the segmentation of the lung cavity using 1H-MRI. iii. A dual-channel, multi-modal segmentation framework was compared to single-channel approaches for estimation of the lung cavity in the domain of HP gas MRI. iv. A hybrid data-driven and model-based approach for the synthesis of HP gas MRI ventilation from CT was compared to approaches utilising DL or computational modelling alone. v. A physics-constrained, multi-channel framework for the synthesis of fully-volumetric ventilation surrogates from 1H-MRI was validated using five-fold cross-validation and an external test data set. Results: i. The 3D CNN, developed via parameterisation experiments, accurately segmented ventilation scans and outperformed conventional ML methods. ii. The 3D CNN produced more accurate segmentations than its 2D analogues for the segmentation of the lung cavity, exhibiting minimal variation in performance between centres, vendors and acquisitions. iii. Dual-channel, multi-modal approaches generate significant improvements compared to methods which use a single imaging modality for the estimation of the lung cavity. iv. The hybrid approach produced synthetic ventilation scans which correlate with HP gas MRI. v. The physics-constrained, 3D multi-channel synthesis framework outperformed approaches which did not integrate computational modelling, demonstrating generalisability to external data. Conclusion: DL approaches demonstrate the ability to segment and synthesise lung MRI across a range of modalities and pulmonary pathologies. These methods outperform computational modelling and classical ML approaches, reducing the time required to adequately edit segmentations and improving the modelling of synthetic ventilation, which may facilitate the clinical translation of DL in structural and functional lung imaging

Quantitative Image Reconstruction Methods for Low Signal-To-Noise Ratio Emission Tomography

Novel internal radionuclide therapies such as radioembolization (RE) with Y-90 loaded microspheres and targeted therapies labeled with Lu-177 offer a unique promise for personalized treatment of cancer because imaging-based pre-treatment dosimetry assessment can be used to determine administered activities, which deliver tumoricidal absorbed doses to lesions while sparing critical organs. At present, however, such therapies are administered with fixed or empiric activities with little or no dosimetry planning. The main reason for lack of dosimetry guided personalized treatment in radionuclide therapies is the challenges and impracticality of quantitative emission tomography imaging and the lack of well established dose-effect relationships, potentially due to inaccuracies in quantitative imaging. While radionuclides for therapy have been chosen for their attractive characteristics for cancer treatment, their suitability for emission tomography imaging is less than ideal. For example, imaging of the almost pure beta emitter, Y-90, involves SPECT via bremsstrahlung photons that have a low and tissue dependent yield or PET via a very low abundance positron emission (32 out of 1 million decays) that leads to a very low true coincidence-rate in the presence of high singles events from bremsstrahlung photons. Lu-177 emits gamma-rays suitable for SPECT, but they are low in intensity (113 keV: 6%, 208 keV: 10%), and only the higher energy emission is generally used because of the large downscatter component associated with the lower energy gamma-ray. The main aim of the research in this thesis is to improve accuracy of quantitative PET and SPECT imaging of therapy radionuclides for dosimetry applications. Although PET is generally considered as superior to SPECT for quantitative imaging, PET imaging of `non-pure' positron emitters can be complex. We focus on quantitative SPECT and PET imaging of two widely used therapy radionuclides, Lu-177 and Y-90, that have challenges associated with low count-rates. The long term goal of our work is to apply the methods we develop to patient imaging for dosimetry based planning to optimize the treatment either before therapy or after each cycle of therapy. For Y-90 PET/CT, we developed an image reconstruction formulation that relaxes the conventional image-domain nonnegativity constraint by instead imposing a positivity constraint on the predicted measurement mean that demonstrated improved quantification in simulated patient studies. For Y-90 SPECT/CT, we propose a new SPECT/CT reconstruction formulation including tissue dependent probabilities for bremsstrahlung generation in the system matrix. In addition to above mentioned quantitative image reconstruction methods specifically developed for each modality in Y-90 imaging, we propose a general image reconstruction method using trained regularizer for low-count PET and SPECT that we test on Y-90 and Lu-177 imaging. Our approach starts with the raw projection data and utilizes trained networks in the iterative image formation process. Specifically, we take a mathematics-based approach where we include convolutional neural networks within the iterative reconstruction process arising from an optimization problem. We further extend the trained regularization method by using anatomical side information. The trained regularizer incorporates the anatomical information using the segmentation mask generated by a trained segmentation network where its input is the co-registered CT image. Overall, the emission tomography methods we have proposed in this work are expected to enhance low-count PET and SPECT imaging of therapy radionuclides in patient studies, which will have value in establishing dose – response relationships and developing imaging based dosimetry guided treatment planning strategies in the future.PHDElectrical and Computer EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155171/1/hongki_1.pd