2,393 research outputs found
A Survey on Deep Learning in Medical Image Analysis
Deep learning algorithms, in particular convolutional networks, have rapidly
become a methodology of choice for analyzing medical images. This paper reviews
the major deep learning concepts pertinent to medical image analysis and
summarizes over 300 contributions to the field, most of which appeared in the
last year. We survey the use of deep learning for image classification, object
detection, segmentation, registration, and other tasks and provide concise
overviews of studies per application area. Open challenges and directions for
future research are discussed.Comment: Revised survey includes expanded discussion section and reworked
introductory section on common deep architectures. Added missed papers from
before Feb 1st 201
Discriminative Representations for Heterogeneous Images and Multimodal Data
Histology images of tumor tissue are an important diagnostic and prognostic tool for pathologists. Recently developed molecular methods group tumors into subtypes to further guide treatment decisions, but they are not routinely performed on all patients. A lower cost and repeatable method to predict tumor subtypes from histology could bring benefits to more cancer patients. Further, combining imaging and genomic data types provides a more complete view of the tumor and may improve prognostication and treatment decisions. While molecular and genomic methods capture the state of a small sample of tumor, histological image analysis provides a spatial view and can identify multiple subtypes in a single tumor. This intra-tumor heterogeneity has yet to be fully understood and its quantification may lead to future insights into tumor progression. In this work, I develop methods to learn appropriate features directly from images using dictionary learning or deep learning. I use multiple instance learning to account for intra-tumor variations in subtype during training, improving subtype predictions and providing insights into tumor heterogeneity. I also integrate image and genomic features to learn a projection to a shared space that is also discriminative. This method can be used for cross-modal classification or to improve predictions from images by also learning from genomic data during training, even if only image data is available at test time.Doctor of Philosoph
Domain Generalization in Computational Pathology: Survey and Guidelines
Deep learning models have exhibited exceptional effectiveness in
Computational Pathology (CPath) by tackling intricate tasks across an array of
histology image analysis applications. Nevertheless, the presence of
out-of-distribution data (stemming from a multitude of sources such as
disparate imaging devices and diverse tissue preparation methods) can cause
\emph{domain shift} (DS). DS decreases the generalization of trained models to
unseen datasets with slightly different data distributions, prompting the need
for innovative \emph{domain generalization} (DG) solutions. Recognizing the
potential of DG methods to significantly influence diagnostic and prognostic
models in cancer studies and clinical practice, we present this survey along
with guidelines on achieving DG in CPath. We rigorously define various DS
types, systematically review and categorize existing DG approaches and
resources in CPath, and provide insights into their advantages, limitations,
and applicability. We also conduct thorough benchmarking experiments with 28
cutting-edge DG algorithms to address a complex DG problem. Our findings
suggest that careful experiment design and CPath-specific Stain Augmentation
technique can be very effective. However, there is no one-size-fits-all
solution for DG in CPath. Therefore, we establish clear guidelines for
detecting and managing DS depending on different scenarios. While most of the
concepts, guidelines, and recommendations are given for applications in CPath,
we believe that they are applicable to most medical image analysis tasks as
well.Comment: Extended Versio
Deep Learning in Medical Image Analysis
The computer-assisted analysis for better interpreting images have been longstanding issues in the medical imaging field. On the image-understanding front, recent advances in machine learning, especially, in the way of deep learning, have made a big leap to help identify, classify, and quantify patterns in medical images. Specifically, exploiting hierarchical feature representations learned solely from data, instead of handcrafted features mostly designed based on domain-specific knowledge, lies at the core of the advances. In that way, deep learning is rapidly proving to be the state-of-the-art foundation, achieving enhanced performances in various medical applications. In this article, we introduce the fundamentals of deep learning methods; review their successes to image registration, anatomical/cell structures detection, tissue segmentation, computer-aided disease diagnosis or prognosis, and so on. We conclude by raising research issues and suggesting future directions for further improvements
Hierarchical Feature Extraction for Efficient Design of Microfluidic Flow Patterns
Deep neural networks are being widely used for feature representation learning in diverse problem areas ranging from object recognition and speech recognition to robotic perception and human disease prediction. We demonstrate a novel, perhaps the first application of deep learning in mechanical design, specifically to learn complex microfluidic flow patterns in order to solve inverse problems in fluid mechanics. A recent discovery showed the ability to control the fluid deformations in a microfluidic channel by placing a sequence of pillars. This provides a fundamental tool for numerous material science, manufacturing and biological applications. However, designing pillar sequences for user-defined deformations is practically infeasible as the current process requires laborious and time-consuming design iterations in a very large, highly nonlinear design space that can have as large as 1015 possibilities. We demonstrate that hierarchical feature extraction can potentially lead to a scalable design tool via learning semantic representations from a relatively small number of flow pattern examples. The paper compares the performances of pre-trained deep neural networks and deep convolutional neural networks as well as their learnt features. We show that a balanced training data generation process with respect to a metric on the output space improves the feature extraction performance. Overall, the deep learning based design process is shown to expedite the current state-of-the-art design approaches by more than 600 times
Magnifying networks for histopathological images with billions of pixels
Amongst the other benefits conferred by the shift from traditional to digital pathology is the potential to use machine learning for diagnosis, prognosis, and personalization. A major challenge in the realization of this potential emerges from the extremely large size of digitized images, which are often in excess of 100,000 × 100,000 pixels. In this paper, we tackle this challenge head-on by diverging from the existing approaches in the literature—which rely on the splitting of the original images into small patches—and introducing magnifying networks (MagNets). By using an attention mechanism, MagNets identify the regions of the gigapixel image that benefit from an analysis on a finer scale. This process is repeated, resulting in an attention-driven coarse-to-fine analysis of only a small portion of the information contained in the original whole-slide images. Importantly, this is achieved using minimal ground truth annotation, namely, using only global, slide-level labels. The results from our tests on the publicly available Camelyon16 and Camelyon17 datasets demonstrate the effectiveness of MagNets—as well as the proposed optimization framework—in the task of whole-slide image classification. Importantly, MagNets process at least five times fewer patches from each whole-slide image than any of the existing end-to-end approaches.Peer reviewe
Deep convolutional neural networks for segmenting 3D in vivo multiphoton images of vasculature in Alzheimer disease mouse models
The health and function of tissue rely on its vasculature network to provide
reliable blood perfusion. Volumetric imaging approaches, such as multiphoton
microscopy, are able to generate detailed 3D images of blood vessels that could
contribute to our understanding of the role of vascular structure in normal
physiology and in disease mechanisms. The segmentation of vessels, a core image
analysis problem, is a bottleneck that has prevented the systematic comparison
of 3D vascular architecture across experimental populations. We explored the
use of convolutional neural networks to segment 3D vessels within volumetric in
vivo images acquired by multiphoton microscopy. We evaluated different network
architectures and machine learning techniques in the context of this
segmentation problem. We show that our optimized convolutional neural network
architecture, which we call DeepVess, yielded a segmentation accuracy that was
better than both the current state-of-the-art and a trained human annotator,
while also being orders of magnitude faster. To explore the effects of aging
and Alzheimer's disease on capillaries, we applied DeepVess to 3D images of
cortical blood vessels in young and old mouse models of Alzheimer's disease and
wild type littermates. We found little difference in the distribution of
capillary diameter or tortuosity between these groups, but did note a decrease
in the number of longer capillary segments () in aged animals as
compared to young, in both wild type and Alzheimer's disease mouse models.Comment: 34 pages, 9 figure
Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directions
Deep learning has seen rapid growth in recent years and achieved
state-of-the-art performance in a wide range of applications. However, training
models typically requires expensive and time-consuming collection of large
quantities of labeled data. This is particularly true within the scope of
medical imaging analysis (MIA), where data are limited and labels are expensive
to be acquired. Thus, label-efficient deep learning methods are developed to
make comprehensive use of the labeled data as well as the abundance of
unlabeled and weak-labeled data. In this survey, we extensively investigated
over 300 recent papers to provide a comprehensive overview of recent progress
on label-efficient learning strategies in MIA. We first present the background
of label-efficient learning and categorize the approaches into different
schemes. Next, we examine the current state-of-the-art methods in detail
through each scheme. Specifically, we provide an in-depth investigation,
covering not only canonical semi-supervised, self-supervised, and
multi-instance learning schemes, but also recently emerged active and
annotation-efficient learning strategies. Moreover, as a comprehensive
contribution to the field, this survey not only elucidates the commonalities
and unique features of the surveyed methods but also presents a detailed
analysis of the current challenges in the field and suggests potential avenues
for future research.Comment: Update Few-shot Method
- …