159 research outputs found
DeepVoting: A Robust and Explainable Deep Network for Semantic Part Detection under Partial Occlusion
In this paper, we study the task of detecting semantic parts of an object,
e.g., a wheel of a car, under partial occlusion. We propose that all models
should be trained without seeing occlusions while being able to transfer the
learned knowledge to deal with occlusions. This setting alleviates the
difficulty in collecting an exponentially large dataset to cover occlusion
patterns and is more essential. In this scenario, the proposal-based deep
networks, like RCNN-series, often produce unsatisfactory results, because both
the proposal extraction and classification stages may be confused by the
irrelevant occluders. To address this, [25] proposed a voting mechanism that
combines multiple local visual cues to detect semantic parts. The semantic
parts can still be detected even though some visual cues are missing due to
occlusions. However, this method is manually-designed, thus is hard to be
optimized in an end-to-end manner.
In this paper, we present DeepVoting, which incorporates the robustness shown
by [25] into a deep network, so that the whole pipeline can be jointly
optimized. Specifically, it adds two layers after the intermediate features of
a deep network, e.g., the pool-4 layer of VGGNet. The first layer extracts the
evidence of local visual cues, and the second layer performs a voting mechanism
by utilizing the spatial relationship between visual cues and semantic parts.
We also propose an improved version DeepVoting+ by learning visual cues from
context outside objects. In experiments, DeepVoting achieves significantly
better performance than several baseline methods, including Faster-RCNN, for
semantic part detection under occlusion. In addition, DeepVoting enjoys
explainability as the detection results can be diagnosed via looking up the
voting cues
Detecting Semantic Parts on Partially Occluded Objects
In this paper, we address the task of detecting semantic parts on partially
occluded objects. We consider a scenario where the model is trained using
non-occluded images but tested on occluded images. The motivation is that there
are infinite number of occlusion patterns in real world, which cannot be fully
covered in the training data. So the models should be inherently robust and
adaptive to occlusions instead of fitting / learning the occlusion patterns in
the training data. Our approach detects semantic parts by accumulating the
confidence of local visual cues. Specifically, the method uses a simple voting
method, based on log-likelihood ratio tests and spatial constraints, to combine
the evidence of local cues. These cues are called visual concepts, which are
derived by clustering the internal states of deep networks. We evaluate our
voting scheme on the VehicleSemanticPart dataset with dense part annotations.
We randomly place two, three or four irrelevant objects onto the target object
to generate testing images with various occlusions. Experiments show that our
algorithm outperforms several competitors in semantic part detection when
occlusions are present.Comment: Accepted to BMVC 2017 (13 pages, 3 figures
G2C: A Generator-to-Classifier Framework Integrating Multi-Stained Visual Cues for Pathological Glomerulus Classification
Pathological glomerulus classification plays a key role in the diagnosis of
nephropathy. As the difference between different subcategories is subtle,
doctors often refer to slides from different staining methods to make
decisions. However, creating correspondence across various stains is
labor-intensive, bringing major difficulties in collecting data and training a
vision-based algorithm to assist nephropathy diagnosis. This paper provides an
alternative solution for integrating multi-stained visual cues for glomerulus
classification. Our approach, named generator-to-classifier (G2C), is a
two-stage framework. Given an input image from a specified stain, several
generators are first applied to estimate its appearances in other staining
methods, and a classifier follows to combine visual cues from different stains
for prediction (whether it is pathological, or which type of pathology it has).
We optimize these two stages in a joint manner. To provide a reasonable
initialization, we pre-train the generators in an unlabeled reference set under
an unpaired image-to-image translation task, and then fine-tune them together
with the classifier. We conduct experiments on a glomerulus type classification
dataset collected by ourselves (there are no publicly available datasets for
this purpose). Although joint optimization slightly harms the authenticity of
the generated patches, it boosts classification performance, suggesting more
effective visual cues are extracted in an automatic way. We also transfer our
model to a public dataset for breast cancer classification, and outperform the
state-of-the-arts significantly.Comment: Accepted by AAAI 201
- …