372 research outputs found
Attribute-Guided Face Generation Using Conditional CycleGAN
We are interested in attribute-guided face generation: given a low-res face
input image, an attribute vector that can be extracted from a high-res image
(attribute image), our new method generates a high-res face image for the
low-res input that satisfies the given attributes. To address this problem, we
condition the CycleGAN and propose conditional CycleGAN, which is designed to
1) handle unpaired training data because the training low/high-res and high-res
attribute images may not necessarily align with each other, and to 2) allow
easy control of the appearance of the generated face via the input attributes.
We demonstrate impressive results on the attribute-guided conditional CycleGAN,
which can synthesize realistic face images with appearance easily controlled by
user-supplied attributes (e.g., gender, makeup, hair color, eyeglasses). Using
the attribute image as identity to produce the corresponding conditional vector
and by incorporating a face verification network, the attribute-guided network
becomes the identity-guided conditional CycleGAN which produces impressive and
interesting results on identity transfer. We demonstrate three applications on
identity-guided conditional CycleGAN: identity-preserving face superresolution,
face swapping, and frontal face generation, which consistently show the
advantage of our new method.Comment: ECCV 201
FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
Over the past few years, we have witnessed the success of deep learning in
image recognition thanks to the availability of large-scale human-annotated
datasets such as PASCAL VOC, ImageNet, and COCO. Although these datasets have
covered a wide range of object categories, there are still a significant number
of objects that are not included. Can we perform the same task without a lot of
human annotations? In this paper, we are interested in few-shot object
segmentation where the number of annotated training examples are limited to 5
only. To evaluate and validate the performance of our approach, we have built a
few-shot segmentation dataset, FSS-1000, which consists of 1000 object classes
with pixelwise annotation of ground-truth segmentation. Unique in FSS-1000, our
dataset contains significant number of objects that have never been seen or
annotated in previous datasets, such as tiny daily objects, merchandise,
cartoon characters, logos, etc. We build our baseline model using standard
backbone networks such as VGG-16, ResNet-101, and Inception. To our surprise,
we found that training our model from scratch using FSS-1000 achieves
comparable and even better results than training with weights pre-trained by
ImageNet which is more than 100 times larger than FSS-1000. Both our approach
and dataset are simple, effective, and easily extensible to learn segmentation
of new object classes given very few annotated training examples. Dataset is
available at https://github.com/HKUSTCV/FSS-1000
Occlusion-Aware Instance Segmentation via BiLayer Network Architectures
Segmenting highly-overlapping image objects is challenging, because there is
typically no distinction between real object contours and occlusion boundaries
on images. Unlike previous instance segmentation methods, we model image
formation as a composition of two overlapping layers, and propose Bilayer
Convolutional Network (BCNet), where the top layer detects occluding objects
(occluders) and the bottom layer infers partially occluded instances
(occludees). The explicit modeling of occlusion relationship with bilayer
structure naturally decouples the boundaries of both the occluding and occluded
instances, and considers the interaction between them during mask regression.
We investigate the efficacy of bilayer structure using two popular
convolutional network designs, namely, Fully Convolutional Network (FCN) and
Graph Convolutional Network (GCN). Further, we formulate bilayer decoupling
using the vision transformer (ViT), by representing instances in the image as
separate learnable occluder and occludee queries. Large and consistent
improvements using one/two-stage and query-based object detectors with various
backbones and network layer choices validate the generalization ability of
bilayer decoupling, as shown by extensive experiments on image instance
segmentation benchmarks (COCO, KINS, COCOA) and video instance segmentation
benchmarks (YTVIS, OVIS, BDD100K MOTS), especially for heavy occlusion cases.
Code and data are available at https://github.com/lkeab/BCNet.Comment: Extended version of "Deep Occlusion-Aware Instance Segmentation with
Overlapping BiLayers", CVPR 2021 (arXiv:2103.12340
- …