41,855 research outputs found
Video retrieval based on deep convolutional neural network
Recently, with the enormous growth of online videos, fast video retrieval
research has received increasing attention. As an extension of image hashing
techniques, traditional video hashing methods mainly depend on hand-crafted
features and transform the real-valued features into binary hash codes. As
videos provide far more diverse and complex visual information than images,
extracting features from videos is much more challenging than that from images.
Therefore, high-level semantic features to represent videos are needed rather
than low-level hand-crafted methods. In this paper, a deep convolutional neural
network is proposed to extract high-level semantic features and a binary hash
function is then integrated into this framework to achieve an end-to-end
optimization. Particularly, our approach also combines triplet loss function
which preserves the relative similarity and difference of videos and
classification loss function as the optimization objective. Experiments have
been performed on two public datasets and the results demonstrate the
superiority of our proposed method compared with other state-of-the-art video
retrieval methods
Convolutional Feature Masking for Joint Object and Stuff Segmentation
The topic of semantic segmentation has witnessed considerable progress due to
the powerful features learned by convolutional neural networks (CNNs). The
current leading approaches for semantic segmentation exploit shape information
by extracting CNN features from masked image regions. This strategy introduces
artificial boundaries on the images and may impact the quality of the extracted
features. Besides, the operations on the raw image domain require to compute
thousands of networks on a single image, which is time-consuming. In this
paper, we propose to exploit shape information via masking convolutional
features. The proposal segments (e.g., super-pixels) are treated as masks on
the convolutional feature maps. The CNN features of segments are directly
masked out from these maps and used to train classifiers for recognition. We
further propose a joint method to handle objects and "stuff" (e.g., grass, sky,
water) in the same framework. State-of-the-art results are demonstrated on
benchmarks of PASCAL VOC and new PASCAL-CONTEXT, with a compelling
computational speed.Comment: IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
201
Controllable Multi-domain Semantic Artwork Synthesis
We present a novel framework for multi-domain synthesis of artwork from
semantic layouts. One of the main limitations of this challenging task is the
lack of publicly available segmentation datasets for art synthesis. To address
this problem, we propose a dataset, which we call ArtSem, that contains 40,000
images of artwork from 4 different domains with their corresponding semantic
label maps. We generate the dataset by first extracting semantic maps from
landscape photography and then propose a conditional Generative Adversarial
Network (GAN)-based approach to generate high-quality artwork from the semantic
maps without necessitating paired training data. Furthermore, we propose an
artwork synthesis model that uses domain-dependent variational encoders for
high-quality multi-domain synthesis. The model is improved and complemented
with a simple but effective normalization method, based on normalizing both the
semantic and style jointly, which we call Spatially STyle-Adaptive
Normalization (SSTAN). In contrast to previous methods that only take semantic
layout as input, our model is able to learn a joint representation of both
style and semantic information, which leads to better generation quality for
synthesizing artistic images. Results indicate that our model learns to
separate the domains in the latent space, and thus, by identifying the
hyperplanes that separate the different domains, we can also perform
fine-grained control of the synthesized artwork. By combining our proposed
dataset and approach, we are able to generate user-controllable artwork that is
of higher quality than existingComment: 15 pages, accepted by CVMJ, to appea
- …