217 research outputs found
Using CycleGANs for effectively reducing image variability across OCT devices and improving retinal fluid segmentation
Optical coherence tomography (OCT) has become the most important imaging
modality in ophthalmology. A substantial amount of research has recently been
devoted to the development of machine learning (ML) models for the
identification and quantification of pathological features in OCT images. Among
the several sources of variability the ML models have to deal with, a major
factor is the acquisition device, which can limit the ML model's
generalizability. In this paper, we propose to reduce the image variability
across different OCT devices (Spectralis and Cirrus) by using CycleGAN, an
unsupervised unpaired image transformation algorithm. The usefulness of this
approach is evaluated in the setting of retinal fluid segmentation, namely
intraretinal cystoid fluid (IRC) and subretinal fluid (SRF). First, we train a
segmentation model on images acquired with a source OCT device. Then we
evaluate the model on (1) source, (2) target and (3) transformed versions of
the target OCT images. The presented transformation strategy shows an F1 score
of 0.4 (0.51) for IRC (SRF) segmentations. Compared with traditional
transformation approaches, this means an F1 score gain of 0.2 (0.12).Comment: * Contributed equally (order was defined by flipping a coin)
--------------- Accepted for publication in the "IEEE International Symposium
on Biomedical Imaging (ISBI) 2019
Deep Representation Learning with Limited Data for Biomedical Image Synthesis, Segmentation, and Detection
Biomedical imaging requires accurate expert annotation and interpretation that can aid medical staff and clinicians in automating differential diagnosis and solving underlying health conditions. With the advent of Deep learning, it has become a standard for reaching expert-level performance in non-invasive biomedical imaging tasks by training with large image datasets. However, with the need for large publicly available datasets, training a deep learning model to learn intrinsic representations becomes harder. Representation learning with limited data has introduced new learning techniques, such as Generative Adversarial Networks, Semi-supervised Learning, and Self-supervised Learning, that can be applied to various biomedical applications. For example, ophthalmologists use color funduscopy (CF) and fluorescein angiography (FA) to diagnose retinal degenerative diseases. However, fluorescein angiography requires injecting a dye, which can create adverse reactions in the patients. So, to alleviate this, a non-invasive technique needs to be developed that can translate fluorescein angiography from fundus images. Similarly, color funduscopy and optical coherence tomography (OCT) are also utilized to semantically segment the vasculature and fluid build-up in spatial and volumetric retinal imaging, which can help with the future prognosis of diseases. Although many automated techniques have been proposed for medical image segmentation, the main drawback is the model's precision in pixel-wise predictions. Another critical challenge in the biomedical imaging field is accurately segmenting and quantifying dynamic behaviors of calcium signals in cells. Calcium imaging is a widely utilized approach to studying subcellular calcium activity and cell function; however, large datasets have yielded a profound need for fast, accurate, and standardized analyses of calcium signals. For example, image sequences from calcium signals in colonic pacemaker cells ICC (Interstitial cells of Cajal) suffer from motion artifacts and high periodic and sensor noise, making it difficult to accurately segment and quantify calcium signal events. Moreover, it is time-consuming and tedious to annotate such a large volume of calcium image stacks or videos and extract their associated spatiotemporal maps. To address these problems, we propose various deep representation learning architectures that utilize limited labels and annotations to address the critical challenges in these biomedical applications. To this end, we detail our proposed semi-supervised, generative adversarial networks and transformer-based architectures for individual learning tasks such as retinal image-to-image translation, vessel and fluid segmentation from fundus and OCT images, breast micro-mass segmentation, and sub-cellular calcium events tracking from videos and spatiotemporal map quantification. We also illustrate two multi-modal multi-task learning frameworks with applications that can be extended to other domains of biomedical applications. The main idea is to incorporate each of these as individual modules to our proposed multi-modal frameworks to solve the existing challenges with 1) Fluorescein angiography synthesis, 2) Retinal vessel and fluid segmentation, 3) Breast micro-mass segmentation, and 4) Dynamic quantification of calcium imaging datasets
Generative adversarial networks in ophthalmology: what are these and how can they be used?
PURPOSE OF REVIEW: The development of deep learning (DL) systems requires a large amount of data, which may be limited by costs, protection of patient information and low prevalence of some conditions. Recent developments in artificial intelligence techniques have provided an innovative alternative to this challenge via the synthesis of biomedical images within a DL framework known as generative adversarial networks (GANs). This paper aims to introduce how GANs can be deployed for image synthesis in ophthalmology and to discuss the potential applications of GANs-produced images. RECENT FINDINGS: Image synthesis is the most relevant function of GANs to the medical field, and it has been widely used for generating 'new' medical images of various modalities. In ophthalmology, GANs have mainly been utilized for augmenting classification and predictive tasks, by synthesizing fundus images and optical coherence tomography images with and without pathologies such as age-related macular degeneration and diabetic retinopathy. Despite their ability to generate high-resolution images, the development of GANs remains data intensive, and there is a lack of consensus on how best to evaluate the outputs produced by GANs. SUMMARY: Although the problem of artificial biomedical data generation is of great interest, image synthesis by GANs represents an innovation with yet unclear relevance for ophthalmology
Interpretable Diabetic Retinopathy Diagnosis based on Biomarker Activation Map
Deep learning classifiers provide the most accurate means of automatically
diagnosing diabetic retinopathy (DR) based on optical coherence tomography
(OCT) and its angiography (OCTA). The power of these models is attributable in
part to the inclusion of hidden layers that provide the complexity required to
achieve a desired task. However, hidden layers also render algorithm outputs
difficult to interpret. Here we introduce a novel biomarker activation map
(BAM) framework based on generative adversarial learning that allows clinicians
to verify and understand classifiers decision-making. A data set including 456
macular scans were graded as non-referable or referable DR based on current
clinical standards. A DR classifier that was used to evaluate our BAM was first
trained based on this data set. The BAM generation framework was designed by
combing two U-shaped generators to provide meaningful interpretability to this
classifier. The main generator was trained to take referable scans as input and
produce an output that would be classified by the classifier as non-referable.
The BAM is then constructed as the difference image between the output and
input of the main generator. To ensure that the BAM only highlights
classifier-utilized biomarkers an assistant generator was trained to do the
opposite, producing scans that would be classified as referable by the
classifier from non-referable scans. The generated BAMs highlighted known
pathologic features including nonperfusion area and retinal fluid. A fully
interpretable classifier based on these highlights could help clinicians better
utilize and verify automated DR diagnosis.Comment: 12 pages, 8 figure
- …