21,856 research outputs found
A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future
As the most fundamental tasks of computer vision, object detection and
segmentation have made tremendous progress in the deep learning era. Due to the
expensive manual labeling, the annotated categories in existing datasets are
often small-scale and pre-defined, i.e., state-of-the-art detectors and
segmentors fail to generalize beyond the closed-vocabulary. To resolve this
limitation, the last few years have witnessed increasing attention toward
Open-Vocabulary Detection (OVD) and Segmentation (OVS). In this survey, we
provide a comprehensive review on the past and recent development of OVD and
OVS. To this end, we develop a taxonomy according to the type of task and
methodology. We find that the permission and usage of weak supervision signals
can well discriminate different methodologies, including: visual-semantic space
mapping, novel visual feature synthesis, region-aware training,
pseudo-labeling, knowledge distillation-based, and transfer learning-based. The
proposed taxonomy is universal across different tasks, covering object
detection, semantic/instance/panoptic segmentation, 3D scene and video
understanding. In each category, its main principles, key challenges,
development routes, strengths, and weaknesses are thoroughly discussed. In
addition, we benchmark each task along with the vital components of each
method. Finally, several promising directions are provided to stimulate future
research
Holistic, Instance-Level Human Parsing
Object parsing -- the task of decomposing an object into its semantic parts
-- has traditionally been formulated as a category-level segmentation problem.
Consequently, when there are multiple objects in an image, current methods
cannot count the number of objects in the scene, nor can they determine which
part belongs to which object. We address this problem by segmenting the parts
of objects at an instance-level, such that each pixel in the image is assigned
a part label, as well as the identity of the object it belongs to. Moreover, we
show how this approach benefits us in obtaining segmentations at coarser
granularities as well. Our proposed network is trained end-to-end given
detections, and begins with a category-level segmentation module. Thereafter, a
differentiable Conditional Random Field, defined over a variable number of
instances for every input image, reasons about the identity of each part by
associating it with a human detection. In contrast to other approaches, our
method can handle the varying number of people in each image and our holistic
network produces state-of-the-art results in instance-level part and human
segmentation, together with competitive results in category-level part
segmentation, all achieved by a single forward-pass through our neural network.Comment: Poster at BMVC 201
- …