32,705 research outputs found
Exploring Context with Deep Structured models for Semantic Segmentation
State-of-the-art semantic image segmentation methods are mostly based on
training deep convolutional neural networks (CNNs). In this work, we proffer to
improve semantic segmentation with the use of contextual information. In
particular, we explore `patch-patch' context and `patch-background' context in
deep CNNs. We formulate deep structured models by combining CNNs and
Conditional Random Fields (CRFs) for learning the patch-patch context between
image regions. Specifically, we formulate CNN-based pairwise potential
functions to capture semantic correlations between neighboring patches.
Efficient piecewise training of the proposed deep structured model is then
applied in order to avoid repeated expensive CRF inference during the course of
back propagation. For capturing the patch-background context, we show that a
network design with traditional multi-scale image inputs and sliding pyramid
pooling is very effective for improving performance. We perform comprehensive
evaluation of the proposed method. We achieve new state-of-the-art performance
on a number of challenging semantic segmentation datasets including ,
-, , -, -,
-, and datasets. Particularly, we report an
intersection-over-union score of on the - dataset.Comment: 16 pages. Accepted to IEEE T. Pattern Analysis & Machine
Intelligence, 2017. Extended version of arXiv:1504.0101
Fast and accurate tumor segmentation of histology images using persistent homology and deep convolutional features
Tumor segmentation in whole-slide images of histology slides is an important step towards computer-assisted diagnosis. In this work, we propose a tumor segmentation framework based on the novel concept of persistent homology profiles (PHPs). For a given image patch, the homology profiles are derived by efficient computation of persistent homology, which is an algebraic tool from homology theory. We propose an efficient way of computing topological persistence of an image, alternative to simplicial homology. The PHPs are devised to distinguish tumor regions from their normal counterparts by modeling the atypical characteristics of tumor nuclei. We propose two variants of our method for tumor segmentation: one that targets speed without compromising accuracy and the other that targets higher accuracy. The fast version is based on the selection of exemplar image patches from a convolution neural network (CNN) and patch classification by quantifying the divergence between the PHPs of exemplars and the input image patch. Detailed comparative evaluation shows that the proposed algorithm is significantly faster than competing algorithms while achieving comparable results. The accurate version combines the PHPs and high-level CNN features and employs a multi-stage ensemble strategy for image patch labeling. Experimental results demonstrate that the combination of PHPs and CNN features outperforms competing algorithms. This study is performed on two independently collected colorectal datasets containing adenoma, adenocarcinoma, signet and healthy cases. Collectively, the accurate tumor segmentation produces the highest average patch-level F1-score, as compared with competing algorithms, on malignant and healthy cases from both the datasets. Overall the proposed framework highlights the utility of persistent homology for histopathology image analysis
Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding
Self-supervised pretraining (SSP) has emerged as a popular technique in
machine learning, enabling the extraction of meaningful feature representations
without labelled data. In the realm of computer vision, pretrained vision
transformers (ViTs) have played a pivotal role in advancing transfer learning.
Nonetheless, the escalating cost of finetuning these large models has posed a
challenge due to the explosion of model size. This study endeavours to evaluate
the effectiveness of pure self-supervised learning (SSL) techniques in computer
vision tasks, obviating the need for finetuning, with the intention of
emulating human-like capabilities in generalisation and recognition of unseen
objects. To this end, we propose an evaluation protocol for zero-shot
segmentation based on a prompting patch. Given a point on the target object as
a prompt, the algorithm calculates the similarity map between the selected
patch and other patches, upon that, a simple thresholding is applied to segment
the target. Another evaluation is intra-object and inter-object similarity to
gauge discriminatory ability of SSP ViTs. Insights from zero-shot segmentation
from prompting and discriminatory abilities of SSP led to the design of a
simple SSP approach, termed MMC. This approaches combines Masked image
modelling for encouraging similarity of local features, Momentum based
self-distillation for transferring semantics from global to local features, and
global Contrast for promoting semantics of global features, to enhance
discriminative representations of SSP ViTs. Consequently, our proposed method
significantly reduces the overlap of intra-object and inter-object
similarities, thereby facilitating effective object segmentation within an
image. Our experiments reveal that MMC delivers top-tier results in zero-shot
semantic segmentation across various datasets
- …