TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency

Accurate segmentation of brain tumors from medical images is important for diagnosis and treatment planning, and it often requires multi-modal or contrast-enhanced images. However, in practice some modalities of a patient may be absent. Synthesizing the missing modality has a potential for filling this gap and achieving high segmentation performance. Existing methods often treat the synthesis and segmentation tasks separately or consider them jointly but without effective regularization of the complex joint model, leading to limited performance. We propose a novel brain Tumor Image Synthesis and Segmentation network (TISS-Net) that obtains the synthesized target modality and segmentation of brain tumors end-to-end with high performance. First, we propose a dual-task-regularized generator that simultaneously obtains a synthesized target modality and a coarse segmentation, which leverages a tumor-aware synthesis loss with perceptibility regularization to minimize the high-level semantic domain gap between synthesized and real target modalities. Based on the synthesized image and the coarse segmentation, we further propose a dual-task segmentor that predicts a refined segmentation and error in the coarse segmentation simultaneously, where a consistency between these two predictions is introduced for regularization. Our TISS-Net was validated with two applications: synthesizing FLAIR images for whole glioma segmentation, and synthesizing contrast-enhanced T1 images for Vestibular Schwannoma segmentation. Experimental results showed that our TISS-Net largely improved the segmentation accuracy compared with direct segmentation from the available modalities, and it outperformed state-of-the-art image synthesis-based segmentation methods

Restauration d'images en IRM anatomique pour l'étude préclinique des marqueurs du vieillissement cérébral

Les maladies neurovasculaires et neurodégénératives liées à l'âge sont en forte augmentation. Alors que ces changements pathologiques montrent des effets sur le cerveau avant l'apparition de symptômes cliniques, une meilleure compréhension du processus de vieillissement normal du cerveau aidera à distinguer l'impact des pathologies connues sur la structure régionale du cerveau. En outre, la connaissance des schémas de rétrécissement du cerveau dans le vieillissement normal pourrait conduire à une meilleure compréhension de ses causes et peut-être à des interventions réduisant la perte de fonctions cérébrales associée à l'atrophie cérébrale. Par conséquent, ce projet de thèse vise à détecter les biomarqueurs du vieillissement normal et pathologique du cerveau dans un modèle de primate non humain, le singe marmouset (Callithrix Jacchus), qui possède des caractéristiques anatomiques plus proches de celles des humains que de celles des rongeurs. Cependant, les changements structurels (par exemple, de volumes, d'épaisseur corticale) qui peuvent se produire au cours de leur vie adulte peuvent être minimes à l'échelle de l'observation. Dans ce contexte, il est essentiel de disposer de techniques d'observation offrant un contraste et une résolution spatiale suffisamment élevés et permettant des évaluations détaillées des changements morphométriques du cerveau associé au vieillissement. Cependant, l'imagerie de petits cerveaux dans une plateforme IRM 3T dédiée à l'homme est une tâche difficile car la résolution spatiale et le contraste obtenus sont insuffisants par rapport à la taille des structures anatomiques observées et à l'échelle des modifications attendues. Cette thèse vise à développer des méthodes de restauration d'image pour les images IRM précliniques qui amélioreront la robustesse des algorithmes de segmentation. L'amélioration de la résolution spatiale des images à un rapport signal/bruit constant limitera les effets de volume partiel dans les voxels situés à la frontière entre deux structures et permettra une meilleure segmentation tout en augmentant la reproductibilité des résultats. Cette étape d'imagerie computationnelle est cruciale pour une analyse morphométrique longitudinale fiable basée sur les voxels et l'identification de marqueurs anatomiques du vieillissement cérébral en suivant les changements de volume dans la matière grise, la matière blanche et le liquide cérébral.Age-related neurovascular and neurodegenerative diseases are increasing significantly. While such pathological changes show effects on the brain before clinical symptoms appear, a better understanding of the normal aging brain process will help distinguish known pathologies' impact on regional brain structure. Furthermore, knowledge of the patterns of brain shrinkage in normal aging could lead to a better understanding of its causes and perhaps to interventions reducing the loss of brain functions. Therefore, this thesis project aims to detect normal and pathological brain aging biomarkers in a non-human primate model, the marmoset monkey (Callithrix Jacchus) which possesses anatomical characteristics more similar to humans than rodents. However, structural changes (e.g., volumes, cortical thickness) that may occur during their adult life may be minimal with respect to the scale of observation. In this context, it is essential to have observation techniques that offer sufficiently high contrast and spatial resolution and allow detailed assessments of the morphometric brain changes associated with aging. However, imaging small brains in a 3T MRI platform dedicated to humans is a challenging task because the spatial resolution and the contrast obtained are insufficient compared to the size of the anatomical structures observed and the scale of the xpected changes with age. This thesis aims to develop image restoration methods for preclinical MR images that will improve the robustness of the segmentation algorithms. Improving the resolution of the images at a constant signal-to-noise ratio will limit the effects of partial volume in voxels located at the border between two structures and allow a better segmentation while increasing the results' reproducibility. This computational imaging step is crucial for a reliable longitudinal voxel-based morphometric analysis and for the identification of anatomical markers of brain aging by following the volume changes in gray matter, white matter and cerebrospinal fluid

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

초고자장 자기공명영상의 B1+ 불균일성 완화를 위한 전자기 퍼텐셜 기반의 하이브리드 모드 성형 연구

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2021. 2. 박남규.Magnetic resonance imaging (MRI) is one of the most popular diagnostic imaging tools with its safety and applicability. By increasing the strength of operating B0, MRI has improved image quality, and recent research has enabled the imaging modality to increase the operating B0 fields over 7T, which is in the ultra-high-field (UHF) range. UHF MRI has various advantages, including the enhancement of the signal-to-noise ratio (SNR), spectral and spatial resolutions, and contrast. Especially, UHF MRI has an irreplaceable strength in precise scanning of the brain tissue to examine various neurological disorders. Nonetheless, the increase of the operating magnetic field causes the severe issue of RF B1+ field inhomogeneity, which is detrimental to homogeneous retrieval of the intensity, SNR, and contrast in MR image. To tackle the critical issue of inhomogeneity, a multitude of approaches for shimming the B1+ inhomogeneity have been proposed. Among them, RF passive shimming by pad structure filled with dielectric materials has proven its validity as a safe and well-compatible solution applicable to clinical applications. While successful in controlling the field distribution, most of the past efforts utilizing the local enhancement of B1+ in the vicinity of the pad structures, especially those in contact with the body, often resulted in deterioration of the global B1+ homogeneity over the ROI. Therefore, a study on a scheme for achieving the global homogeneity of B1+ is required. In this dissertation, we propose the notion of the mode shaping based on the evanescent coupling of the electromagnetic potentials to address the issue of B1+ field homogeneity. Treating the human head as a resonator, we apply an auxiliary potential well evanescent coupled to the head potential, to investigate the effects of the auxiliary potential configuration on the mode shaping and the resultant field homogenization. From the analysis and optimization, we obtain a robust mode shaping pad solution to achieve 2D global homogenization of axial B1+ field for the phantom of various geometry and the realistic voxel model of heterogeneous materials, which is applicable to the conventional 2D MRI scanning. Furthermore, extending the mode shaping approach with symmetry breaking, we propose the mode shaping solutions for 3D global homogenization of B1+ field. For the practical assessment of the feasibility of the mode shaping solutions, the SAR and robustness analysis of the solutions are also conducted. We believe that this study will expand the capability of the RF passive shimming in UHF MRI by providing an unconventional viewpoint and systematic guideline for the mitigation of B1+ inhomogeneity.자기 공명 영상법은 안정성과 확장성을 바탕으로 가장 광범위하게 이용되고 있는 영상 기법 중 하나이다. 자기 공명 영상법은 동작 정 자기장을 높임으로써 영상의 질을 향상시킬 수 있는데, 최근에는 7T 이상의 초고자장 자기 공명 영상법이 활용되고 있다. 초고자장 자기 공명 영상법은 신호 대비 잡음도, 공간 시간 해상도, 대조도 등을 향상시키는데, 이를 통해 특히 뇌 정밀 영상 촬영에 대체 불가한 강점을 가진다. 이러한 장점에도 불구하고 정 자기장의 증가는 고주파 신호 B1+ 필드의 불균일성을 야기하며, 이는 다시 이미지 퀄리티를 떨어뜨리는 등 원치 않는 영향으로 이어진다. B1+ 필드 불균일성 문제를 해결하기 위해 다양한 방법이 시도되었으며, 이 중에서도 유전체 물질로 채운 구조물인 패드를 이용한 수동 보정 접근은 기존 시스템에 호환이 되고 안정성을 인정받아 임상에서의 적용 가능성이 알려져 있는 대표적인 전략이다. 필드 패턴을 바꾸어주는 효과를 기반으로 하여 관심 영역에 붙여 주변부의 불균일성을 성공적으로 완화하는 이 방법은 최근 고 유전체 물질의 활용과 더불어 관심을 받고 있지만, 관심 부분 영역 전체에 대해서는 악영향을 수반하므로 광역 균일화를 위한 방법에 대한 연구가 필요하다. 본 학위 논문에서는, B1+ 필드의 광역 균일화를 위한 방법으로 퍼텐셜의 에바네센트 커플링을 기반으로 한 모드 성형 방식을 제안한다. 인체를 공진기로 보고 그것과 에바네센트 결합을 하는 보조 퍼텐셜을 적용함으로써 모드 성형 능력을 확인하였으며, 그것을 조절함으로써 스캐닝하려는 대상의 형태나 물질 분포에 강건한 축성 B1+ 필드의 광역 균일화를 구현할 수 있다. 또한, 해당 개념을 확장하여 삼차원 광역 균일화를 수행하는 모드 성형법을 제안한다. 이와 더불어, 이러한 모드 성형법의 실제적 적용가능성을 평가하기 위해 SAR와 강건성 분석을 수행하였다. 본 연구는 초고자장 자기 공명 영상법의 B1+ 불균일성 완화에 대한 색다른 시각과 체계적인 방법을 제공함으로써 고주파 신호 균일화에의 수동 보정 방법의 역할을 확장하는 지침이 될 것으로 기대된다.Abstract i Table of Contents iv List of Tables viii List of Figures ix Chapter 1. Introduction 1 1.1 Ultra-high-field magnetic resonance imaging: promising scheme for clinical imaging 2 1.2 Inhomogeneity problem in UHF MRI: Motivation 5 1.3 Dissertation overview 7 Chapter 2. Theory and method for the B1+ shimming 9 2.1 Electromagnetics in the UHF MRI 10 2.1.1 Principal physics of MRI system in view of electromagnetics 10 2.1.2 Issue of RF B1+ field inhomogeneity in UHF MRI 14 2.2 B1 shimming in UHF MRI 17 2.2.1 Current approaches and achievements for B1+ shimming 17 2.2.2 Background and motivation of our strategy for B1 shimming: mode shaping pad 21 2.3 Optical mode shaping based on evanescent coupling for mitigation of B1+ inhomogeneity 23 2.3.1 UHF MRI systems as an optical waveguide 23 2.3.2 Mode shaping via evanescent coupling in optics 24 2.3.3 Evanescent coupling of electromagnetic potentials in UHF MRI 26 2.4 Conclusion 29 Chapter 3. Hybrid mode shaping with auxiliary EM potential for global 2D homogenization 30 3.1 Mode shaping for 2D MRI scanning 31 3.2 Concept of hybrid mode shaping with auxiliary EM potential 34 3.3 Optimization process 38 3.4 Effect of the phantom and pad geometry and other material parameters of the pad 47 3.5 Effect of the inhomogeneous distribution of materials: human voxel model 52 3.6 Effect of the mode shaping potential pad on the SAR distributions 56 3.7 Robustness and stability of the mode shaping solution 59 3.8 Conclusion 61 Chapter 4. Hybrid mode shaping with auxiliary EM potential for global 3D homogenization 63 4.1 Mode shaping for 3D MRI scanning 64 4.2 Hat pad potential for lower-order mode excitation 66 4.3 Asymmetric shifted pad potential 72 4.4 Effect of the shifted potential pad on the SAR distribution 79 4.5 Robustness of the mode shaping with asymmetric potential pad 81 4.6 Conclusion 83 Chapter 5. Conclusion 84 Appendix A. Supplements for Chapter 3 86 A.1 Material and geometry for the MIDA voxel model 86 A.2 Excitation with realistic TEM coils 91 Appendix B. Supplements for Chapter 4 93 B.1 Cylinder can solution for the global homogenization 93 Appendix C 97 Bibliography 98 Abstract in Korean 109Docto

Automated Diagnosis of Cardiovascular Diseases from Cardiac Magnetic Resonance Imaging Using Deep Learning Models: A Review

In recent years, cardiovascular diseases (CVDs) have become one of the leading causes of mortality globally. CVDs appear with minor symptoms and progressively get worse. The majority of people experience symptoms such as exhaustion, shortness of breath, ankle swelling, fluid retention, and other symptoms when starting CVD. Coronary artery disease (CAD), arrhythmia, cardiomyopathy, congenital heart defect (CHD), mitral regurgitation, and angina are the most common CVDs. Clinical methods such as blood tests, electrocardiography (ECG) signals, and medical imaging are the most effective methods used for the detection of CVDs. Among the diagnostic methods, cardiac magnetic resonance imaging (CMR) is increasingly used to diagnose, monitor the disease, plan treatment and predict CVDs. Coupled with all the advantages of CMR data, CVDs diagnosis is challenging for physicians due to many slices of data, low contrast, etc. To address these issues, deep learning (DL) techniques have been employed to the diagnosis of CVDs using CMR data, and much research is currently being conducted in this field. This review provides an overview of the studies performed in CVDs detection using CMR images and DL techniques. The introduction section examined CVDs types, diagnostic methods, and the most important medical imaging techniques. In the following, investigations to detect CVDs using CMR images and the most significant DL methods are presented. Another section discussed the challenges in diagnosing CVDs from CMR data. Next, the discussion section discusses the results of this review, and future work in CVDs diagnosis from CMR images and DL techniques are outlined. The most important findings of this study are presented in the conclusion section

Reasoning with Uncertainty in Deep Learning for Safer Medical Image Computing

Deep learning is now ubiquitous in the research field of medical image computing. As such technologies progress towards clinical translation, the question of safety becomes critical. Once deployed, machine learning systems unavoidably face situations where the correct decision or prediction is ambiguous. However, the current methods disproportionately rely on deterministic algorithms, lacking a mechanism to represent and manipulate uncertainty. In safety-critical applications such as medical imaging, reasoning under uncertainty is crucial for developing a reliable decision making system. Probabilistic machine learning provides a natural framework to quantify the degree of uncertainty over different variables of interest, be it the prediction, the model parameters and structures, or the underlying data (images and labels). Probability distributions are used to represent all the uncertain unobserved quantities in a model and how they relate to the data, and probability theory is used as a language to compute and manipulate these distributions. In this thesis, we explore probabilistic modelling as a framework to integrate uncertainty information into deep learning models, and demonstrate its utility in various high-dimensional medical imaging applications. In the process, we make several fundamental enhancements to current methods. We categorise our contributions into three groups according to the types of uncertainties being modelled: (i) predictive; (ii) structural and (iii) human uncertainty. Firstly, we discuss the importance of quantifying predictive uncertainty and understanding its sources for developing a risk-averse and transparent medical image enhancement application. We demonstrate how a measure of predictive uncertainty can be used as a proxy for the predictive accuracy in the absence of ground-truths. Furthermore, assuming the structure of the model is flexible enough for the task, we introduce a way to decompose the predictive uncertainty into its orthogonal sources i.e. aleatoric and parameter uncertainty. We show the potential utility of such decoupling in providing a quantitative “explanations” into the model performance. Secondly, we introduce our recent attempts at learning model structures directly from data. One work proposes a method based on variational inference to learn a posterior distribution over connectivity structures within a neural network architecture for multi-task learning, and share some preliminary results in the MR-only radiotherapy planning application. Another work explores how the training algorithm of decision trees could be extended to grow the architecture of a neural network to adapt to the given availability of data and the complexity of the task. Lastly, we develop methods to model the “measurement noise” (e.g., biases and skill levels) of human annotators, and integrate this information into the learning process of the neural network classifier. In particular, we show that explicitly modelling the uncertainty involved in the annotation process not only leads to an improvement in robustness to label noise, but also yields useful insights into the patterns of errors that characterise individual experts