15 research outputs found
Weakly Supervised Multi-Task Learning for Cell Detection and Segmentation
Cell detection and segmentation is fundamental for all downstream analysis of
digital pathology images. However, obtaining the pixel-level ground truth for
single cell segmentation is extremely labor intensive. To overcome this
challenge, we developed an end-to-end deep learning algorithm to perform both
single cell detection and segmentation using only point labels. This is
achieved through the combination of different task orientated point label
encoding methods and a multi-task scheduler for training. We apply and validate
our algorithm on PMS2 stained colon rectal cancer and tonsil tissue images.
Compared to the state-of-the-art, our algorithm shows significant improvement
in cell detection and segmentation without increasing the annotation efforts
Domain Adaptation-based Augmentation for Weakly Supervised Nuclei Detection
The detection of nuclei is one of the most fundamental components of
computational pathology. Current state-of-the-art methods are based on deep
learning, with the prerequisite that extensive labeled datasets are available.
The increasing number of patient cohorts to be analyzed, the diversity of
tissue stains and indications, as well as the cost of dataset labeling
motivates the development of novel methods to reduce labeling effort across
domains. We introduce in this work a weakly supervised 'inter-domain' approach
that (i) performs stain normalization and unpaired image-to-image translation
to transform labeled images on a source domain to synthetic labeled images on
an unlabeled target domain and (ii) uses the resulting synthetic labeled images
to train a detection network on the target domain. Extensive experiments show
the superiority of the proposed approach against the state-of-the-art
'intra-domain' detection based on fully-supervised learning
Scribble2Label: Scribble-Supervised Cell Segmentation via Self-Generating Pseudo-Labels with Consistency
Segmentation is a fundamental process in microscopic cell image analysis.
With the advent of recent advances in deep learning, more accurate and
high-throughput cell segmentation has become feasible. However, most existing
deep learning-based cell segmentation algorithms require fully annotated
ground-truth cell labels, which are time-consuming and labor-intensive to
generate. In this paper, we introduce Scribble2Label, a novel weakly-supervised
cell segmentation framework that exploits only a handful of scribble
annotations without full segmentation labels. The core idea is to combine
pseudo-labeling and label filtering to generate reliable labels from weak
supervision. For this, we leverage the consistency of predictions by
iteratively averaging the predictions to improve pseudo labels. We demonstrate
the performance of Scribble2Label by comparing it to several state-of-the-art
cell segmentation methods with various cell image modalities, including
bright-field, fluorescence, and electron microscopy. We also show that our
method performs robustly across different levels of scribble details, which
confirms that only a few scribble annotations are required in real-use cases.Comment: MICCAI 2020 accepte
Weakly-Supervised Segmentation for Disease Localization in Chest X-Ray Images
Deep Convolutional Neural Networks have proven effective in solving the task
of semantic segmentation. However, their efficiency heavily relies on the
pixel-level annotations that are expensive to get and often require domain
expertise, especially in medical imaging. Weakly supervised semantic
segmentation helps to overcome these issues and also provides explainable deep
learning models. In this paper, we propose a novel approach to the semantic
segmentation of medical chest X-ray images with only image-level class labels
as supervision. We improve the disease localization accuracy by combining three
approaches as consecutive steps. First, we generate pseudo segmentation labels
of abnormal regions in the training images through a supervised classification
model enhanced with a regularization procedure. The obtained activation maps
are then post-processed and propagated into a second classification
model-Inter-pixel Relation Network, which improves the boundaries between
different object classes. Finally, the resulting pseudo-labels are used to
train a proposed fully supervised segmentation model. We analyze the robustness
of the presented method and test its performance on two distinct datasets:
PASCAL VOC 2012 and SIIM-ACR Pneumothorax. We achieve significant results in
the segmentation on both datasets using only image-level annotations. We show
that this approach is applicable to chest X-rays for detecting an anomalous
volume of air in the pleural space between the lung and the chest wall. Our
code has been made publicly available.Comment: Accepted to AIME 202
Bounding boxes for weakly supervised segmentation: Global constraints get close to full supervision
We propose a novel weakly supervised learning segmentation based on several
global constraints derived from box annotations. Particularly, we leverage a
classical tightness prior to a deep learning setting via imposing a set of
constraints on the network outputs. Such a powerful topological prior prevents
solutions from excessive shrinking by enforcing any horizontal or vertical line
within the bounding box to contain, at least, one pixel of the foreground
region. Furthermore, we integrate our deep tightness prior with a global
background emptiness constraint, guiding training with information outside the
bounding box. We demonstrate experimentally that such a global constraint is
much more powerful than standard cross-entropy for the background class. Our
optimization problem is challenging as it takes the form of a large set of
inequality constraints on the outputs of deep networks. We solve it with
sequence of unconstrained losses based on a recent powerful extension of the
log-barrier method, which is well-known in the context of interior-point
methods. This accommodates standard stochastic gradient descent (SGD) for
training deep networks, while avoiding computationally expensive and unstable
Lagrangian dual steps and projections. Extensive experiments over two different
public data sets and applications (prostate and brain lesions) demonstrate that
the synergy between our global tightness and emptiness priors yield very
competitive performances, approaching full supervision and outperforming
significantly DeepCut. Furthermore, our approach removes the need for
computationally expensive proposal generation. Our code is shared anonymously.Comment: Full paper, accepted for presentation at MIDL202
ACCL: Adversarial constrained-CNN loss for weakly supervised medical image segmentation
We propose adversarial constrained-CNN loss, a new paradigm of
constrained-CNN loss methods, for weakly supervised medical image segmentation.
In the new paradigm, prior knowledge is encoded and depicted by reference
masks, and is further employed to impose constraints on segmentation outputs
through adversarial learning with reference masks. Unlike pseudo label methods
for weakly supervised segmentation, such reference masks are used to train a
discriminator rather than a segmentation network, and thus are not required to
be paired with specific images. Our new paradigm not only greatly facilitates
imposing prior knowledge on network's outputs, but also provides stronger and
higher-order constraints, i.e., distribution approximation, through adversarial
learning. Extensive experiments involving different medical modalities,
different anatomical structures, different topologies of the object of
interest, different levels of prior knowledge and weakly supervised annotations
with different annotation ratios is conducted to evaluate our ACCL method.
Consistently superior segmentation results over the size constrained-CNN loss
method have been achieved, some of which are close to the results of full
supervision, thus fully verifying the effectiveness and generalization of our
method. Specifically, we report an average Dice score of 75.4% with an average
annotation ratio of 0.65%, surpassing the prior art, i.e., the size
constrained-CNN loss method, by a large margin of 11.4%. Our codes are made
publicly available at https://github.com/PengyiZhang/ACCL
Switching Loss for Generalized Nucleus Detection in Histopathology
The accuracy of deep learning methods for two foundational tasks in medical
image analysis -- detection and segmentation -- can suffer from class
imbalance. We propose a `switching loss' function that adaptively shifts the
emphasis between foreground and background classes. While the existing loss
functions to address this problem were motivated by the classification task,
the switching loss is based on Dice loss, which is better suited for
segmentation and detection. Furthermore, to get the most out the training
samples, we adapt the loss with each mini-batch, unlike previous proposals that
adapt once for the entire training set. A nucleus detector trained using the
proposed loss function on a source dataset outperformed those trained using
cross-entropy, Dice, or focal losses. Remarkably, without retraining on target
datasets, our pre-trained nucleus detector also outperformed existing nucleus
detectors that were trained on at least some of the images from the target
datasets. To establish a broad utility of the proposed loss, we also confirmed
that it led to more accurate ventricle segmentation in MRI as compared to the
other loss functions. Our GPU-enabled pre-trained nucleus detection software is
also ready to process whole slide images right out-of-the-box and is usably
fast
NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images
Object segmentation is an important step in the workflow of computational
pathology. Deep learning based models generally require large amount of labeled
data for precise and reliable prediction. However, collecting labeled data is
expensive because it often requires expert knowledge, particularly in medical
imaging domain where labels are the result of a time-consuming analysis made by
one or more human experts. As nuclei, cells and glands are fundamental objects
for downstream analysis in computational pathology/cytology, in this paper we
propose a simple CNN-based approach to speed up collecting annotations for
these objects which requires minimum interaction from the annotator. We show
that for nuclei and cells in histology and cytology images, one click inside
each object is enough for NuClick to yield a precise annotation. For
multicellular structures such as glands, we propose a novel approach to provide
the NuClick with a squiggle as a guiding signal, enabling it to segment the
glandular boundaries. These supervisory signals are fed to the network as
auxiliary inputs along with RGB channels. With detailed experiments, we show
that NuClick is adaptable to the object scale, robust against variations in the
user input, adaptable to new domains, and delivers reliable annotations. An
instance segmentation model trained on masks generated by NuClick achieved the
first rank in LYON19 challenge. As exemplar outputs of our framework, we are
releasing two datasets: 1) a dataset of lymphocyte annotations within IHC
images, and 2) a dataset of segmented WBCs in blood smear images
Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images
Nuclei segmentation is a fundamental task in histopathology image analysis.
Typically, such segmentation tasks require significant effort to manually
generate accurate pixel-wise annotations for fully supervised training. To
alleviate such tedious and manual effort, in this paper we propose a novel
weakly supervised segmentation framework based on partial points annotation,
i.e., only a small portion of nuclei locations in each image are labeled. The
framework consists of two learning stages. In the first stage, we design a
semi-supervised strategy to learn a detection model from partially labeled
nuclei locations. Specifically, an extended Gaussian mask is designed to train
an initial model with partially labeled data. Then, selftraining with
background propagation is proposed to make use of the unlabeled regions to
boost nuclei detection and suppress false positives. In the second stage, a
segmentation model is trained from the detected nuclei locations in a
weakly-supervised fashion. Two types of coarse labels with complementary
information are derived from the detected points and are then utilized to train
a deep neural network. The fully-connected conditional random field loss is
utilized in training to further refine the model without introducing extra
computational complexity during inference. The proposed method is extensively
evaluated on two nuclei segmentation datasets. The experimental results
demonstrate that our method can achieve competitive performance compared to the
fully supervised counterpart and the state-of-the-art methods while requiring
significantly less annotation effort.Comment: 12 page
Semantic annotation for computational pathology: Multidisciplinary experience and best practice recommendations
Recent advances in whole slide imaging (WSI) technology have led to the
development of a myriad of computer vision and artificial intelligence (AI)
based diagnostic, prognostic, and predictive algorithms. Computational
Pathology (CPath) offers an integrated solution to utilize information embedded
in pathology WSIs beyond what we obtain through visual assessment. For
automated analysis of WSIs and validation of machine learning (ML) models,
annotations at the slide, tissue and cellular levels are required. The
annotation of important visual constructs in pathology images is an important
component of CPath projects. Improper annotations can result in algorithms
which are hard to interpret and can potentially produce inaccurate and
inconsistent results. Despite the crucial role of annotations in CPath
projects, there are no well-defined guidelines or best practices on how
annotations should be carried out. In this paper, we address this shortcoming
by presenting the experience and best practices acquired during the execution
of a large-scale annotation exercise involving a multidisciplinary team of
pathologists, ML experts and researchers as part of the Pathology image data
Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a
real-world case study along with examples of different types of annotations,
diagnostic algorithm, annotation data dictionary and annotation constructs. The
analyses reported in this work highlight best practice recommendations that can
be used as annotation guidelines over the lifecycle of a CPath project