3,682 research outputs found
Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects
Medical imaging analysis has witnessed remarkable advancements even
surpassing human-level performance in recent years, driven by the rapid
development of advanced deep-learning algorithms. However, when the inference
dataset slightly differs from what the model has seen during one-time training,
the model performance is greatly compromised. The situation requires restarting
the training process using both the old and the new data which is
computationally costly, does not align with the human learning process, and
imposes storage constraints and privacy concerns. Alternatively, continual
learning has emerged as a crucial approach for developing unified and
sustainable deep models to deal with new classes, tasks, and the drifting
nature of data in non-stationary environments for various application areas.
Continual learning techniques enable models to adapt and accumulate knowledge
over time, which is essential for maintaining performance on evolving datasets
and novel tasks. This systematic review paper provides a comprehensive overview
of the state-of-the-art in continual learning techniques applied to medical
imaging analysis. We present an extensive survey of existing research, covering
topics including catastrophic forgetting, data drifts, stability, and
plasticity requirements. Further, an in-depth discussion of key components of a
continual learning framework such as continual learning scenarios, techniques,
evaluation schemes, and metrics is provided. Continual learning techniques
encompass various categories, including rehearsal, regularization,
architectural, and hybrid strategies. We assess the popularity and
applicability of continual learning categories in various medical sub-fields
like radiology and histopathology..
Data efficient deep learning for medical image analysis: A survey
The rapid evolution of deep learning has significantly advanced the field of
medical image analysis. However, despite these achievements, the further
enhancement of deep learning models for medical image analysis faces a
significant challenge due to the scarcity of large, well-annotated datasets. To
address this issue, recent years have witnessed a growing emphasis on the
development of data-efficient deep learning methods. This paper conducts a
thorough review of data-efficient deep learning methods for medical image
analysis. To this end, we categorize these methods based on the level of
supervision they rely on, encompassing categories such as no supervision,
inexact supervision, incomplete supervision, inaccurate supervision, and only
limited supervision. We further divide these categories into finer
subcategories. For example, we categorize inexact supervision into multiple
instance learning and learning with weak annotations. Similarly, we categorize
incomplete supervision into semi-supervised learning, active learning, and
domain-adaptive learning and so on. Furthermore, we systematically summarize
commonly used datasets for data efficient deep learning in medical image
analysis and investigate future research directions to conclude this survey.Comment: Under Revie
Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives
Deep learning has demonstrated remarkable performance across various tasks in
medical imaging. However, these approaches primarily focus on supervised
learning, assuming that the training and testing data are drawn from the same
distribution. Unfortunately, this assumption may not always hold true in
practice. To address these issues, unsupervised domain adaptation (UDA)
techniques have been developed to transfer knowledge from a labeled domain to a
related but unlabeled domain. In recent years, significant advancements have
been made in UDA, resulting in a wide range of methodologies, including feature
alignment, image translation, self-supervision, and disentangled representation
methods, among others. In this paper, we provide a comprehensive literature
review of recent deep UDA approaches in medical imaging from a technical
perspective. Specifically, we categorize current UDA research in medical
imaging into six groups and further divide them into finer subcategories based
on the different tasks they perform. We also discuss the respective datasets
used in the studies to assess the divergence between the different domains.
Finally, we discuss emerging areas and provide insights and discussions on
future research directions to conclude this survey.Comment: Under Revie
NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research
We introduce the Never Ending VIsual-classification Stream (NEVIS'22), a
benchmark consisting of a stream of over 100 visual classification tasks,
sorted chronologically and extracted from papers sampled uniformly from
computer vision proceedings spanning the last three decades. The resulting
stream reflects what the research community thought was meaningful at any point
in time. Despite being limited to classification, the resulting stream has a
rich diversity of tasks from OCR, to texture analysis, crowd counting, scene
recognition, and so forth. The diversity is also reflected in the wide range of
dataset sizes, spanning over four orders of magnitude. Overall, NEVIS'22 poses
an unprecedented challenge for current sequential learning approaches due to
the scale and diversity of tasks, yet with a low entry barrier as it is limited
to a single modality and each task is a classical supervised learning problem.
Moreover, we provide a reference implementation including strong baselines and
a simple evaluation protocol to compare methods in terms of their trade-off
between accuracy and compute. We hope that NEVIS'22 can be useful to
researchers working on continual learning, meta-learning, AutoML and more
generally sequential learning, and help these communities join forces towards
more robust and efficient models that efficiently adapt to a never ending
stream of data. Implementations have been made available at
https://github.com/deepmind/dm_nevis
A Continual Learning Approach for Cross-Domain White Blood Cell Classification
Accurate classification of white blood cells in peripheral blood is essential
for diagnosing hematological diseases. Due to constantly evolving clinical
settings, data sources, and disease classifications, it is necessary to update
machine learning classification models regularly for practical real-world use.
Such models significantly benefit from sequentially learning from incoming data
streams without forgetting previously acquired knowledge. However, models can
suffer from catastrophic forgetting, causing a drop in performance on previous
tasks when fine-tuned on new data. Here, we propose a rehearsal-based continual
learning approach for class incremental and domain incremental scenarios in
white blood cell classification. To choose representative samples from previous
tasks, we employ exemplar set selection based on the model's predictions. This
involves selecting the most confident samples and the most challenging samples
identified through uncertainty estimation of the model. We thoroughly evaluated
our proposed approach on three white blood cell classification datasets that
differ in color, resolution, and class composition, including scenarios where
new domains or new classes are introduced to the model with every task. We also
test a long class incremental experiment with both new domains and new classes.
Our results demonstrate that our approach outperforms established baselines in
continual learning, including existing iCaRL and EWC methods for classifying
white blood cells in cross-domain environments.Comment: Accepted for publication at workshop on Domain Adaptation and
Representation Transfer (DART) in International Conference on Medical Image
Computing and Computer Assisted Intervention (MICCAI 2023
Applications of Sequential Learning for Medical Image Classification
Purpose: The aim of this work is to develop a neural network training
framework for continual training of small amounts of medical imaging data and
create heuristics to assess training in the absence of a hold-out validation or
test set.
Materials and Methods: We formulated a retrospective sequential learning
approach that would train and consistently update a model on mini-batches of
medical images over time. We address problems that impede sequential learning
such as overfitting, catastrophic forgetting, and concept drift through PyTorch
convolutional neural networks (CNN) and publicly available Medical MNIST and
NIH Chest X-Ray imaging datasets. We begin by comparing two methods for a
sequentially trained CNN with and without base pre-training. We then transition
to two methods of unique training and validation data recruitment to estimate
full information extraction without overfitting. Lastly, we consider an example
of real-life data that shows how our approach would see mainstream research
implementation.
Results: For the first experiment, both approaches successfully reach a ~95%
accuracy threshold, although the short pre-training step enables sequential
accuracy to plateau in fewer steps. The second experiment comparing two methods
showed better performance with the second method which crosses the ~90%
accuracy threshold much sooner. The final experiment showed a slight advantage
with a pre-training step that allows the CNN to cross ~60% threshold much
sooner than without pre-training.
Conclusion: We have displayed sequential learning as a serviceable
multi-classification technique statistically comparable to traditional CNNs
that can acquire data in small increments feasible for clinically realistic
scenarios
- …