From Fully-Supervised, Single-Task to Scarcely-Supervised, Multi-Task Deep Learning for Medical Image Analysis

Image analysis based on machine learning has gained prominence with the advent of deep learning, particularly in medical imaging. To be effective in addressing challenging image analysis tasks, however, conventional deep neural networks require large corpora of annotated training data, which are unfortunately scarce in the medical domain, thus often rendering fully-supervised learning strategies ineffective.This thesis devises for use in a variety of medical image analysis applications a series of novel deep learning methods, ranging from fully-supervised, single-task learning to scarcely-supervised, multi-task learning that makes efficient use of annotated training data. Specifically, its main contributions include (1) fully-supervised, single-task learning for the segmentation of pulmonary lobes from chest CT scans and the analysis of scoliosis from spine X-ray images; (2) supervised, single-task, domain-generalized pulmonary segmentation in chest X-ray images and retinal vasculature segmentation in fundoscopic images; (3) largely-unsupervised, multiple-task learning via deep generative modeling for the joint synthesis and classification of medical image data; and (4) partly-supervised, multiple-task learning for the combined segmentation and classification of chest and spine X-ray images

An In-Depth Statistical Review of Retinal Image Processing Models from a Clinical Perspective

The burgeoning field of retinal image processing is critical in facilitating early diagnosis and treatment of retinal diseases, which are amongst the leading causes of vision impairment globally. Despite rapid advancements, existing machine learning models for retinal image processing are characterized by significant limitations, including disparities in pre-processing, segmentation, and classification methodologies, as well as inconsistencies in post-processing operations. These limitations hinder the realization of accurate, reliable, and clinically relevant outcomes. This paper provides an in-depth statistical review of extant machine learning models used in retinal image processing, meticulously comparing them based on their internal operating characteristics and performance levels. By adopting a robust analytical approach, our review delineates the strengths and weaknesses of current models, offering comprehensive insights that are instrumental in guiding future research and development in this domain. Furthermore, this review underscores the potential clinical impacts of these models, highlighting their pivotal role in enhancing diagnostic accuracy, prognostic assessments, and therapeutic interventions for retinal disorders. In conclusion, our work not only bridges the existing knowledge gap in the literature but also paves the way for the evolution of more sophisticated and clinically-aligned retinal image processing models, ultimately contributing to improved patient outcomes and advancements in ophthalmic care

Fluorescence microscopy image analysis of retinal neurons using deep learning

An essential goal of neuroscience is to understand the brain by simultaneously identifying, measuring, and analyzing activity from individual cells within a neural population in live brain tissue. Analyzing fluorescence microscopy (FM) images in real-time with computational algorithms is essential for achieving this goal. Deep learning techniques have shown promise in this area, but face domain-specific challenges due to limited training data, significant amounts of voxel noise in FM images, and thin structures present in large 3D images. In this thesis, I address these issues by introducing a novel deep learning pipeline to analyze static FM images of neurons with minimal data requirements and demonstrate the pipeline’s ability to segment neurons from low signal-to-noise ratio FM images with few training samples. The first step of this pipeline employs a Generative Adversarial Network (GAN) equipped to learn imaging properties from a small set of static FM images acquired for a given neuroscientific experiment. Operating like an actual microscope, our fully-trained GAN can then generate realistic static FM images from volumetric reconstructions of neurons with added control over the intensity and noise of the generated images. For the second step in our pipeline, a novel segmentation network is trained on GAN-generated images with reconstructed neurons serving as “gold standard” ground truths. While training on a large dataset of FM images is optimal, a 15\% improvement in neuron segmentation accuracy from noisy FM images is shown when architectures are fine-tuned only on a small subsample of real image data. To evaluate the overall feasibility of our pipeline and the utility of generated images, 2 novel synthetic and 3 newly acquired FM image datasets are introduced along with a new evaluation protocol for FM image ”realness” that incorporates content, texture, and expert opinion metrics. While this pipeline's primary application is to segment neurons from highly noisy FM images, its utility can be extended to automate other FM tasks such as synapse identification, neuron classification, or super-resolution

A refined equilibrium generative adversarial network for retinal vessel segmentation

OBJECTIVE: Retinal vessel morphological parameters are vital indicator for early diagnosis of ophthalmological diseases and cardiovascular events. However, segmentation performance is highly influenced by elusive vessels, especially in low-contrast background and lesion regions. In this work, we present an end-to-end synthetic neural network to strengthen elusive vessels segmentation capability, containing a symmetric equilibrium generative adversarial network (SEGAN), multi-scale features refine blocks (MSFRB), and attention mechanism (AM). METHOD: The proposed network is superior in detail information extraction by maximizing multi-scale features representation. First, SEGAN constructs a symmetric adversarial architecture in which generator is forced to produce more realistic images with local details. Second, MSFRB are devised to optimize the feature merging process, thereby maximally maintaining high resolution information. Finally, the AM is employed to encourage the network to concentrate on discriminative features. RESULTS: On public dataset DRIVE, STARE, CHASEDB1, and HRF, we evaluate our network quantitatively and compare it with state-of-the-art works. The ablation experiment shows that SEGAN, MSFRB, and AM both contribute to the desirable performance. Conclusion: The proposed network outperforms the existing methods and effectively functions in elusive vessels segmentation, achieving highest scores in Sensitivity, G-Mean, Precision, and F1-Score while maintaining the top level in other metrics. Significance: The satisfactory performance and computational efficiency offer great potential in clinical retinal vessel segmentation application. Meanwhile, the network could be utilized to extract detail information in other biomedical image computing

Fundus2Angio: A Conditional GAN Architecture for Generating Fluorescein Angiography Images from Retinal Fundus Photography

Carrying out clinical diagnosis of retinal vascular degeneration using Fluorescein Angiography (FA) is a time consuming process and can pose significant adverse effects on the patient. Angiography requires insertion of a dye that may cause severe adverse effects and can even be fatal. Currently, there are no non-invasive systems capable of generating Fluorescein Angiography images. However, retinal fundus photography is a non-invasive imaging technique that can be completed in a few seconds. In order to eliminate the need for FA, we propose a conditional generative adversarial network (GAN) to translate fundus images to FA images. The proposed GAN consists of a novel residual block capable of generating high quality FA images. These images are important tools in the differential diagnosis of retinal diseases without the need for invasive procedure with possible side effects. Our experiments show that the proposed architecture outperforms other state-of-the-art generative networks. Furthermore, our proposed model achieves better qualitative results indistinguishable from real angiograms.Comment: 14 pages, Accepted to 15th International Symposium on Visual Computing 202

Human treelike tubular structure segmentation: A comprehensive review and future perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed