Deep Learning with Limited Labels for Medical Imaging

Recent advancements in deep learning-based AI technologies provide an automatic tool to revolutionise medical image computing. Training a deep learning model requires a large amount of labelled data. Acquiring labels for medical images is extremely challenging due to the high cost in terms of both money and time, especially for the pixel-wise segmentation task of volumetric medical scans. However, obtaining unlabelled medical scans is relatively easier compared to acquiring labels for those images. This work addresses the pervasive issue of limited labels in training deep learning models for medical imaging. It begins by exploring different strategies of entropy regularisation in the joint training of labelled and unlabelled data to reduce the time and cost associated with manual labelling for medical image segmentation. Of particular interest are consistency regularisation and pseudo labelling. Specifically, this work proposes a well-calibrated semi-supervised segmentation framework that utilises consistency regularisation on different morphological feature perturbations, representing a significant step towards safer AI in medical imaging. Furthermore, it reformulates pseudo labelling in semi-supervised learning as an Expectation-Maximisation framework. Building upon this new formulation, the work explains the empirical successes of pseudo labelling and introduces a generalisation of the technique, accompanied by variational inference to learn its true posterior distribution. The applications of pseudo labelling in segmentation tasks are also presented. Lastly, this work explores unsupervised deep learning for parameter estimation of diffusion MRI signals, employing a hierarchical variational clustering framework and representation learning

How good is good enough? Strategies for dealing with unreliable segmentation annotations of medical data

Medical image segmentation is an essential topic in computer vision and medical image analysis, because it enables the precise and accurate segmentation of organs and lesions for healthcare applications. Deep learning has dominated in medical image segmentation due to increasingly powerful computational resources, successful neural network architecture engineering, and access to large amounts of medical imaging data with high-quality annotations. However, annotating medical imaging data is time-consuming and expensive, and sometimes the annotations are unreliable. This DPhil thesis presents a comprehensive study that explores deep learning techniques in medical image segmentation under various challenging situations of unreliable medical imaging data. These situations include: (1) conventional supervised learning to tackle comprehensive data annotation with full dense masks, (2) semi-supervised learning to tackle partial data annotation with full dense masks, (3) noise-robust learning to tackle comprehensive data annotation with noisy dense masks, and (4) weakly-supervised learning to tackle comprehensive data annotation with sketchy contours for network training. The proposed medical image segmentation strategies improve deep learning techniques to effectively address a series of challenges in medical image analysis, including limited annotated data, noisy annotations, and sparse annotations. These advancements aim to bring deep learning techniques of medical image analysis into practical clinical scenarios. By overcoming these challenges, the strategies establish a more robust and reliable application of deep learning methods which is valuable for improving diagnostic precision and patient care outcomes in real-world clinical environments

딥러닝 방법론을 이용한 높은 적용성을 가진 수경재배 파프리카 대상 절차 기반 모델 개발

학위논문(박사) -- 서울대학교대학원 : 농업생명과학대학 농림생물자원학부, 2022. 8. 손정익.Many agricultural challenges are entangled in a complex interaction between crops and the environment. As a simplifying tool, crop modeling is a process of abstracting and interpreting agricultural phenomena. Understanding based on this interpretation can play a role in supporting academic and social decisions in agriculture. Process-based crop models have solved the challenges for decades to enhance the productivity and quality of crop production; the remaining objectives have led to demand for crop models handling multidirectional analyses with multidimensional information. As a possible milestone to satisfy this goal, deep learning algorithms have been introduced to the complicated tasks in agriculture. However, the algorithms could not replace existing crop models because of the research fragmentation and low accessibility of the crop models. This study established a developmental protocol for a process-based crop model with deep learning methodology. Literature Review introduced deep learning and crop modeling, and it explained the reasons for the necessity of this protocol despite numerous deep learning applications for agriculture. Base studies were conducted with several greenhouse data in Chapters 1 and 2: transfer learning and U-Net structure were utilized to construct an infrastructure for the deep learning application; HyperOpt, a Bayesian optimization method, was tested to calibrate crop models to compare the existing crop models with the developed model. Finally, the process-based crop model with full deep neural networks, DeepCrop, was developed with an attention mechanism and multitask decoders for hydroponic sweet peppers (Capsicum annuum var. annuum) in Chapter 3. The methodology for data integrity showed adequate accuracy, so it was applied to the data in all chapters. HyperOpt was able to calibrate food and feed crop models for sweet peppers. Therefore, the compared models in the final chapter were optimized using HyperOpt. DeepCrop was trained to simulate several growth factors with environment data. The trained DeepCrop was evaluated with unseen data, and it showed the highest modeling efficiency (=0.76) and the lowest normalized root mean squared error (=0.18) than the compared models. With the high adaptability of DeepCrop, it can be used for studies on various scales and purposes. Since all methods adequately solved the given tasks and underlay the DeepCrop development, the established protocol can be a high throughput for enhancing accessibility of crop models, resulting in unifying crop modeling studies.농업 시스템에서 발생하는 문제들은 작물과 환경의 상호작용 하에 복잡하게 얽혀 있다. 작물 모델링은 대상을 단순화하는 방법으로써, 농업에서 일어나는 현상을 추상화하고 해석하는 과정이다. 모델링을 통해 대상을 이해하는 것은 농업 분야의 학술적 및 사회적 결정을 지원할 수 있다. 지난 수년 간 절차 기반 작물 모델은 농업의 문제들을 해결하여 작물 생산성 및 품질을 증진시켰으며, 현재 작물 모델링에 남아있는 과제들은 다차원 정보를 다방향에서 분석할 수 있는 작물 모델을 필요로 하게 되었다. 이를 만족시킬 수 있는 지침으로써, 복잡한 농업적 과제들을 목표로 딥러닝 알고리즘이 도입되었다. 그러나, 이 알고리즘들은 낮은 데이터 완결성 및 높은 연구 다양성 때문에 기존의 작물 모델들을 대체하지는 못했다. 본 연구에서는 딥러닝 방법론을 이용하여 절차 기반 작물 모델을 구축하는 개발 프로토콜을 확립하였다. Literature Review에서는 딥러닝과 작물 모델에 대해 소개하고, 농업으로의 딥러닝 적용 연구가 많음에도 이 프로토콜이 필요한 이유를 설명하였다. 제1장과 2장에서는 국내 여러 지역의 데이터를 이용하여 전이 학습 및 U-Net 구조를 활용하여 딥러닝 모델 적용을 위한 기반을 마련하고, 베이지안 최적화 방법인 HyperOpt를 사용하여 기존 모델과 딥러닝 기반 모델을 비교하기 위해 시험적으로 WOFOST 작물 모델을 보정하는 등 모델 개발을 위한 기반 연구를 수행하였다. 마지막으로, 제3장에서는 주의 메커니즘 및 다중 작업 디코더를 가진 완전 심층 신경망 절차 기반 작물 모델인 DeepCrop을 수경재배 파프리카(Capsicum annuum var. annuum) 대상으로 개발하였다. 데이터 완결성을 위한 기술들은 적합한 정확도를 보여주었으며, 전체 챕터 데이터에 적용하였다. HyperOpt는 식량 및 사료 작물 모델들을 파프리카 대상으로 보정할 수 있었다. 따라서, 제3장의 비교 대상 모델들에 대해 HyperOpt를 사용하였다. DeepCrop은 환경 데이터를 이용하고 여러 생육 지표를 예측하도록 학습되었다. 학습에 사용하지 않은 데이터를 이용하여 학습된 DeepCrop를 평가하였으며, 이 때 비교 모델들 중 가장 높은 모형 효율(EF=0.76)과 가장 낮은 표준화 평균 제곱근 오차(NRMSE=0.18)를 보여주었다. DeepCrop은 높은 적용성을 기반으로 다양한 범위와 목적을 가진 연구에 사용될 수 있을 것이다. 모든 방법들이 주어진 작업을 적절히 풀어냈고 DeepCrop 개발의 근거가 되었으므로, 본 논문에서 확립한 프로토콜은 작물 모델의 접근성을 향상시킬 수 있는 획기적인 방향을 제시하였고, 작물 모델 연구의 통합에 기여할 수 있을 것으로 기대한다.LITERATURE REVIEW 1 ABSTRACT 1 BACKGROUND 3 REMARKABLE APPLICABILITY AND ACCESSIBILITY OF DEEP LEARNING 12 DEEP LEARNING APPLICATIONS FOR CROP PRODUCTION 17 THRESHOLDS TO APPLY DEEP LEARNING TO CROP MODELS 18 NECESSITY TO PRIORITIZE DEEP-LEARNING-BASED CROP MODELS 20 REQUIREMENTS OF THE DEEP-LEARNING-BASED CROP MODELS 21 OPENING REMARKS AND THESIS OBJECTIVES 22 LITERATURE CITED 23 Chapter 1 34 Chapter 1-1 35 ABSTRACT 35 INTRODUCTION 37 MATERIALS AND METHODS 40 RESULTS 50 DISCUSSION 59 CONCLUSION 63 LITERATURE CITED 64 Chapter 1-2 71 ABSTRACT 71 INTRODUCTION 73 MATERIALS AND METHODS 75 RESULTS 84 DISCUSSION 92 CONCLUSION 101 LITERATURE CITED 102 Chapter 2 108 ABSTRACT 108 NOMENCLATURE 110 INTRODUCTION 112 MATERIALS AND METHODS 115 RESULTS 124 DISCUSSION 133 CONCLUSION 137 LITERATURE CITED 138 Chapter 3 144 ABSTRACT 144 INTRODUCTION 146 MATERIALS AND METHODS 149 RESULTS 169 DISCUSSION 182 CONCLUSION 187 LITERATURE CITED 188 GENERAL DISCUSSION 196 GENERAL CONCLUSION 201 ABSTRACT IN KOREAN 203 APPENDIX 204박

Medical Image Segmentation with Deep Convolutional Neural Networks

Medical imaging is the technique and process of creating visual representations of the body of a patient for clinical analysis and medical intervention. Healthcare professionals rely heavily on medical images and image documentation for proper diagnosis and treatment. However, manual interpretation and analysis of medical images are time-consuming, and inaccurate when the interpreter is not well-trained. Fully automatic segmentation of the region of interest from medical images has been researched for years to enhance the efficiency and accuracy of understanding such images. With the advance of deep learning, various neural network models have gained great success in semantic segmentation and sparked research interests in medical image segmentation using deep learning. We propose three convolutional frameworks to segment tissues from different types of medical images. Comprehensive experiments and analyses are conducted on various segmentation neural networks to demonstrate the effectiveness of our methods. Furthermore, datasets built for training our networks and full implementations are published

DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds

We present a new deep unfolding network for analysis-sparsity-based Compressed Sensing. The proposed network coined Decoding Network (DECONET) jointly learns a decoder that reconstructs vectors from their incomplete, noisy measurements and a redundant sparsifying analysis operator, which is shared across the layers of DECONET. Moreover, we formulate the hypothesis class of DECONET and estimate its associated Rademacher complexity. Then, we use this estimate to deliver meaningful upper bounds for the generalization error of DECONET. Finally, the validity of our theoretical results is assessed and comparisons to state-of-the-art unfolding networks are made, on both synthetic and real-world datasets. Experimental results indicate that our proposed network outperforms the baselines, consistently for all datasets, and its behaviour complies with our theoretical findings.Comment: Accepted in IEEE Transactions on Signal Processin

Exploring probabilistic models for semi-supervised learning

Deep neural networks are increasingly harnessed for computer vision tasks, thanks to their robust performance. However, their training demands large-scale labeled datasets, which are labor-intensive to prepare. Semi-supervised learning (SSL) offers a solution by learning from a mix of labeled and unlabeled data. While most state-of-the-art SSL methods follow a deterministic approach, the exploration of their probabilistic counterparts remains limited. This research area is important because probabilistic models can provide uncertainty estimates critical for real-world applications. For instance, SSL-trained models may fall short of those trained with supervised learning due to potential pseudo-label errors in unlabeled data, and these models are more likely to make wrong predictions in practice. Especially in critical sectors like medical image analysis and autonomous driving, decision-makers must understand the model’s limitations and when incorrect predictions may occur, insights often provided by uncertainty estimates. Furthermore, uncertainty can also serve as a criterion for filtering out unreliable pseudo-labels when unlabeled samples are used for training, potentially improving deep model performance. This thesis furthers the exploration of probabilistic models for SSL. Drawing on the widely-used Bayesian approximation tool, Monte Carlo (MC) dropout, I propose a new probabilistic framework, the Generative Bayesian Deep Learning (GBDL) architecture, for semi-supervised medical image segmentation. This approach not only mitigates potential overfitting found in previous methods but also achieves superior results across four evaluation metrics. Unlike its empirically designed predecessors, GBDL is underpinned by a full Bayesian formulation, providing a theoretical probabilistic foundation. Acknowledging MC dropout’s limitations, I introduce NP-Match, a novel proba- bilistic approach for large-scale semi-supervised image classification. We evaluated NP-Match’s generalization capabilities through extensive experiments in different challenging settings such as standard, imbalanced, and multi-label semi-supervised image classification. According to the experimental results, NP-Match not only competes favorably with previous state-of-the-art methods but also estimates uncertainty more rapidly than MC-dropout-based models, thus enhancing both training and testing efficiency. Lastly, I propose NP-SemiSeg, a new probabilistic model for semi-supervised se- mantic segmentation. This flexible model can be integrated with various existing segmentation frameworks to make predictions and estimate uncertainty. Experiments indicate that NP-SemiSeg surpasses MC dropout in accuracy, uncertainty quantification, and speed

The roles of online and offline replay in planning

Animals and humans replay neural patterns encoding trajectories through their environment, both whilst they solve decision-making tasks and during rest. Both on-task and off-task replay are believed to contribute to flexible decision making, though how their relative contributions differ remains unclear. We investigated this question by using magnetoencephalography (MEG) to study human subjects while they performed a decision-making task that was designed to reveal the decision algorithms employed. We characterised subjects in terms of how flexibly each adjusted their choices to changes in temporal, spatial and reward structure. The more flexible a subject, the more they replayed trajectories during task performance, and this replay was coupled with re-planning of the encoded trajectories. The less flexible a subject, the more they replayed previously preferred trajectories during rest periods between task epochs. The data suggest that online and offline replay both participate in planning but support distinct decision strategies