Task adapted reconstruction for inverse problems

The paper considers the problem of performing a task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and task as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any task that is encodable as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation

Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs)

Positron emission tomography (PET) image synthesis plays an important role, which can be used to boost the training data for computer aided diagnosis systems. However, existing image synthesis methods have problems in synthesizing the low resolution PET images. To address these limitations, we propose multi-channel generative adversarial networks (M-GAN) based PET image synthesis method. Different to the existing methods which rely on using low-level features, the proposed M-GAN is capable to represent the features in a high-level of semantic based on the adversarial learning concept. In addition, M-GAN enables to take the input from the annotation (label) to synthesize the high uptake regions e.g., tumors and from the computed tomography (CT) images to constrain the appearance consistency and output the synthetic PET images directly. Our results on 50 lung cancer PET-CT studies indicate that our method was much closer to the real PET images when compared with the existing methods.Comment: 9 pages, 2 figure

해부학적 유도 PET 재구성: 매끄럽지 않은 사전 함수부터 딥러닝 접근까지

학위논문 (박사) -- 서울대학교 대학원 : 의과대학 의과학과, 2021. 2. 이재성.Advances in simultaneous positron emission tomography/magnetic resonance imaging (PET/MRI) technology have led to an active investigation of the anatomy-guided regularized PET image reconstruction algorithm based on MR images. Among the various priors proposed for anatomy-guided regularized PET image reconstruction, Bowsher’s method based on second-order smoothing priors sometimes suffers from over-smoothing of detailed structures. Therefore, in this study, we propose a Bowsher prior based on the l1 norm and an iteratively reweighting scheme to overcome the limitation of the original Bowsher method. In addition, we have derived a closed solution for iterative image reconstruction based on this non-smooth prior. A comparison study between the original l2 and proposed l1 Bowsher priors were conducted using computer simulation and real human data. In the simulation and real data application, small lesions with abnormal PET uptake were better detected by the proposed l1 Bowsher prior methods than the original Bowsher prior. The original l2 Bowsher leads to a decreased PET intensity in small lesions when there is no clear separation between the lesions and surrounding tissue in the anatomical prior. However, the proposed l1 Bowsher prior methods showed better contrast between the tumors and surrounding tissues owing to the intrinsic edge-preserving property of the prior which is attributed to the sparseness induced by l1 norm, especially in the iterative reweighting scheme. Besides, the proposed methods demonstrated lower bias and less hyper-parameter dependency on PET intensity estimation in the regions with matched anatomical boundaries in PET and MRI. Moreover, based on the formulation of l1 Bowsher prior, the unrolled network containing the conventional maximum-likelihood expectation-maximization (ML-EM) module was also proposed. The convolutional layers successfully learned the distribution of anatomically-guided PET images and the EM module corrected the intermediate outputs by comparing them with sinograms. The proposed unrolled network showed better performance than ordinary U-Net, where the regional uptake is less biased and deviated. Therefore, these methods will help improve the PET image quality based on the anatomical side information.양전자방출단층촬영 / 자기공명영상 (PET/MRI) 동시 획득 기술의 발전으로 MR 영상을 기반으로 한 해부학적 사전 함수로 정규화 된 PET 영상 재구성 알고리즘에 대한 심도있는 평가가 이루어졌다. 해부학 기반으로 정규화 된 PET 이미지 재구성을 위해 제안 된 다양한 사전 중 2차 평활화 사전함수에 기반한 Bowsher의 방법은 때때로 세부 구조의 과도한 평활화로 어려움을 겪는다. 따라서 본 연구에서는 원래 Bowsher 방법의 한계를 극복하기 위해 l1 norm에 기반한 Bowsher 사전 함수와 반복적인 재가중치 기법을 제안한다. 또한, 우리는 이 매끄럽지 않은 사전 함수를 이용한 반복적 이미지 재구성에 대해 닫힌 해를 도출했다. 원래 l2와 제안 된 l1 Bowsher 사전 함수 간의 비교 연구는 컴퓨터 시뮬레이션과 실제 데이터를 사용하여 수행되었다. 시뮬레이션 및 실제 데이터에서 비정상적인 PET 흡수를 가진 작은 병변은 원래 Bowsher 이전보다 제안 된 l1 Bowsher 사전 방법으로 더 잘 감지되었다. 원래의 l2 Bowsher는 해부학적 영상에서 병변과 주변 조직 사이에 명확한 분리가 없을 때 작은 병변에서의 PET 강도를 감소시킨다. 그러나 제안 된 l1 Bowsher 사전 방법은 특히 반복적 재가중치 기법에서 l1 노름에 의해 유도된 희소성에 기인한 특성으로 인해 종양과 주변 조직 사이에 더 나은 대비를 보여주었다. 또한 제안된 방법은 PET과 MRI의 해부학적 경계가 일치하는 영역에서 PET 강도 추정에 대한 편향이 더 낮고 하이퍼 파라미터 종속성이 적음을 보여주었다. 또한, l1Bowsher 사전 함수의 닫힌 해를 기반으로 기존의 ML-EM (maximum-likelihood expectation-maximization) 모듈을 포함하는 펼쳐진 네트워크도 제안되었다. 컨볼루션 레이어는 해부학적으로 유도 재구성된 PET 이미지의 분포를 성공적으로 학습했으며, EM 모듈은 중간 출력들을 사이노그램과 비교하여 결과 이미지가 잘 들어맞게 수정했다. 제안된 펼쳐진 네트워크는 지역의 흡수선량이 덜 편향되고 편차가 적어, 일반 U-Net보다 더 나은 성능을 보여주었다. 따라서 이러한 방법들은 해부학적 정보를 기반으로 PET 이미지 품질을 향상시키는 데 유용할 것이다.Chapter 1. Introduction 1 1.1. Backgrounds 1 1.1.1. Positron Emission Tomography 1 1.1.2. Maximum a Posterior Reconstruction 1 1.1.3. Anatomical Prior 2 1.1.4. Proposed l_1 Bowsher Prior 3 1.1.5. Deep Learning for MR-less Application 4 1.2. Purpose of the Research 4 Chapter 2. Anatomically-guided PET Reconstruction Using Bowsher Prior 6 2.1. Backgrounds 6 2.1.1. PET Data Model 6 2.1.2. Original Bowsher Prior 7 2.2. Methods and Materials 8 2.2.1. Proposed l_1 Bowsher Prior 8 2.2.2. Iterative Reweighting 13 2.2.3. Computer Simulations 15 2.2.4. Human Data 16 2.2.5. Image Analysis 17 2.3. Results 19 2.3.1. Simulation with Brain Phantom 19 2.3.2.Human Data 20 2.4. Discussions 25 Chapter 3. Deep Learning Approach for Anatomically-guided PET Reconstruction 31 3.1. Backgrounds 31 3.2. Methods and Materials 33 3.2.1. Douglas-Rachford Splitting 33 3.2.2. Network Architecture 34 3.2.3. Dataset and Training Details 35 3.2.4. Image Analysis 36 3.3. Results 37 3.4. Discussions 38 Chapter 4. Conclusions 40 Bibliography 41 Abstract in Korean (국문 초록) 52Docto

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Potentials and caveats of AI in Hybrid Imaging

State-of-the-art patient management frequently mandates the investigation of both anatomy and physiology of the patients. Hybrid imaging modalities such as the PET/MRI, PET/CT and SPECT/CT have the ability to provide both structural and functional information of the investigated tissues in a single examination. With the introduction of such advanced hardware fusion, new problems arise such as the exceedingly large amount of multi-modality data that requires novel approaches of how to extract a maximum of clinical information from large sets of multi-dimensional imaging data. Artificial intelligence (AI) has emerged as one of the leading technologies that has shown promise in facilitating highly integrative analysis of multi-parametric data. Specifically, the usefulness of AI algorithms in the medical imaging field has been heavily investigated in the realms of (1) image acquisition and reconstruction, (2) post-processing and (3) data mining and modelling. Here, we aim to provide an overview of the challenges encountered in hybrid imaging and discuss how AI algorithms can facilitate potential solutions. In addition, we highlight the pitfalls and challenges in using advanced AI algorithms in the context of hybrid imaging and provide suggestions for building robust AI solutions that enable reproducible and transparent research

Artificial Intelligence Techniques in Medical Imaging: A Systematic Review

This scientific review presents a comprehensive overview of medical imaging modalities and their diverse applications in artificial intelligence (AI)-based disease classification and segmentation. The paper begins by explaining the fundamental concepts of AI, machine learning (ML), and deep learning (DL). It provides a summary of their different types to establish a solid foundation for the subsequent analysis. The prmary focus of this study is to conduct a systematic review of research articles that examine disease classification and segmentation in different anatomical regions using AI methodologies. The analysis includes a thorough examination of the results reported in each article, extracting important insights and identifying emerging trends. Moreover, the paper critically discusses the challenges encountered during these studies, including issues related to data availability and quality, model generalization, and interpretability. The aim is to provide guidance for optimizing technique selection. The analysis highlights the prominence of hybrid approaches, which seamlessly integrate ML and DL techniques, in achieving effective and relevant results across various disease types. The promising potential of these hybrid models opens up new opportunities for future research in the field of medical diagnosis. Additionally, addressing the challenges posed by the limited availability of annotated medical images through the incorporation of medical image synthesis and transfer learning techniques is identified as a crucial focus for future research efforts

A novel framework for efficient identification of brain cancer region from brain MRI

Diagnosis of brain cancer using existing imaging techniques, e.g., Magnetic Resonance Imaging (MRI) is shrouded with various degrees of challenges. At present, there are very few significant research models focusing on introducing some novel and unique solutions towards such problems of detection. Moreover, existing techniques are found to have lesser accuracy as compared to other detection schemes. Therefore, the proposed paper presents a framework that introduces a series of simple and computationally cost-effective techniques that have assisted in leveraging the accuracy level to a very higher degree. The proposed framework takes the input image and subjects it to non-conventional segmentation mechanism followed by optimizing the performance using directed acyclic graph, Bayesian Network, and neural network. The study outcome of the proposed system shows the significantly higher degree of accuracy in detection performance as compared to frequently existing approaches

Mathematics of biomedical imaging today—a perspective

Biomedical imaging is a fascinating, rich and dynamic research area, which has huge importance in biomedical research and clinical practice alike. The key technology behind the processing, and automated analysis and quantification of imaging data is mathematics. Starting with the optimisation of the image acquisition and the reconstruction of an image from indirect tomographic measurement data, all the way to the automated segmentation of tumours in medical images and the design of optimal treatment plans based on image biomarkers, mathematics appears in all of these in different flavours. Non-smooth optimisation in the context of sparsity-promoting image priors, partial differential equations for image registration and motion estimation, and deep neural networks for image segmentation, to name just a few. In this article, we present and review mathematical topics that arise within the whole biomedical imaging pipeline, from tomographic measurements to clinical support tools, and highlight some modern topics and open problems. The article is addressed to both biomedical researchers who want to get a taste of where mathematics arises in biomedical imaging as well as mathematicians who are interested in what mathematical challenges biomedical imaging research entails