6 research outputs found
ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes
Exploiting synthetic data to learn deep models has attracted increasing
attention in recent years. However, the intrinsic domain difference between
synthetic and real images usually causes a significant performance drop when
applying the learned model to real world scenarios. This is mainly due to two
reasons: 1) the model overfits to synthetic images, making the convolutional
filters incompetent to extract informative representation for real images; 2)
there is a distribution difference between synthetic and real data, which is
also known as the domain adaptation problem. To this end, we propose a new
reality oriented adaptation approach for urban scene semantic segmentation by
learning from synthetic data. First, we propose a target guided distillation
approach to learn the real image style, which is achieved by training the
segmentation model to imitate a pretrained real style model using real images.
Second, we further take advantage of the intrinsic spatial structure presented
in urban scene images, and propose a spatial-aware adaptation scheme to
effectively align the distribution of two domains. These two modules can be
readily integrated with existing state-of-the-art semantic segmentation
networks to improve their generalizability when adapting from synthetic to real
urban scenes. We evaluate the proposed method on Cityscapes dataset by adapting
from GTAV and SYNTHIA datasets, where the results demonstrate the effectiveness
of our method.Comment: Add experiments on SYNTHIA, CVPR 2018 camera-ready versio
Image Segmentation using Sparse Subset Selection
In this paper, we present a new image segmentation method based on the
concept of sparse subset selection. Starting with an over-segmentation, we
adopt local spectral histogram features to encode the visual information of the
small segments into high-dimensional vectors, called superpixel features. Then,
the superpixel features are fed into a novel convex model which efficiently
leverages the features to group the superpixels into a proper number of
coherent regions. Our model automatically determines the optimal number of
coherent regions and superpixels assignment to shape final segments. To solve
our model, we propose a numerical algorithm based on the alternating direction
method of multipliers (ADMM), whose iterations consist of two highly
parallelizable sub-problems. We show each sub-problem enjoys closed-form
solution which makes the ADMM iterations computationally very efficient.
Extensive experiments on benchmark image segmentation datasets demonstrate that
our proposed method in combination with an over-segmentation can provide high
quality and competitive results compared to the existing state-of-the-art
methods.Comment: IEEE Winter Conference on Applications of Computer Vision (WACV),
201
Domain Adaptive Faster R-CNN for Object Detection in the Wild
Object detection typically assumes that training and test data are drawn from
an identical distribution, which, however, does not always hold in practice.
Such a distribution mismatch will lead to a significant performance drop. In
this work, we aim to improve the cross-domain robustness of object detection.
We tackle the domain shift on two levels: 1) the image-level shift, such as
image style, illumination, etc, and 2) the instance-level shift, such as object
appearance, size, etc. We build our approach based on the recent
state-of-the-art Faster R-CNN model, and design two domain adaptation
components, on image level and instance level, to reduce the domain
discrepancy. The two domain adaptation components are based on H-divergence
theory, and are implemented by learning a domain classifier in adversarial
training manner. The domain classifiers on different levels are further
reinforced with a consistency regularization to learn a domain-invariant region
proposal network (RPN) in the Faster R-CNN model. We evaluate our newly
proposed approach using multiple datasets including Cityscapes, KITTI, SIM10K,
etc. The results demonstrate the effectiveness of our proposed approach for
robust object detection in various domain shift scenarios.Comment: Accepted to CVPR 201
Scale-aware alignment of hierarchical image segmentation
© 2016 IEEE. Image segmentation is a key component in many computer vision systems, and it is recovering a prominent spot in the literature as methods improve and overcome their limitations. The outputs of most recent algorithms are in the form of a hierarchical segmentation, which provides segmentation at different scales in a single tree-like structure. Commonly, these hierarchical methods start from some low-level features, and are not aware of the scale information of the different regions in them. As such, one might need to work on many different levels of the hierarchy to find the objects in the scene. This work tries to modify the existing hierarchical algorithm by improving their alignment, that is, by trying to modify the depth of the regions in the tree to better couple depth and scale. To do so, we first train a regressor to predict the scale of regions using mid-level features. We then define the anchor slice as the set of regions that better balance between over-segmentation and under-segmentation. The output of our method is an improved hierarchy, re-aligned by the anchor slice. To demonstrate the power of our method, we perform comprehensive experiments, which show that our method, as a post-processing step, can significantly improve the quality of the hierarchical segmentation representations, and ease the usage of hierarchical image segmentation to high-level vision tasks such as object segmentation. We also prove that the improvement generalizes well across different algorithms and datasets, with a low computational cost.1Chen Y., Dai D., Pont-Tuset J., Van Gool L., ''Scale-aware alignment of hierarchical image segmentation'', 29th IEEE conference on computer vision and pattern recognition - CVPR 2016, 9 pp., June 26 - July 1, 2016, Las Vegas, Nevada, USA.status: publishe