57 research outputs found
Manifold Constrained Low-Rank Decomposition
Low-rank decomposition (LRD) is a state-of-the-art method for visual data
reconstruction and modelling. However, it is a very challenging problem when
the image data contains significant occlusion, noise, illumination variation,
and misalignment from rotation or viewpoint changes. We leverage the specific
structure of data in order to improve the performance of LRD when the data are
not ideal. To this end, we propose a new framework that embeds manifold priors
into LRD. To implement the framework, we design an alternating direction method
of multipliers (ADMM) method which efficiently integrates the manifold
constraints during the optimization process. The proposed approach is
successfully used to calculate low-rank models from face images, hand-written
digits and planar surface images. The results show a consistent increase of
performance when compared to the state-of-the-art over a wide range of
realistic image misalignments and corruptions
Attention-Aware Face Hallucination via Deep Reinforcement Learning
Face hallucination is a domain-specific super-resolution problem with the
goal to generate high-resolution (HR) faces from low-resolution (LR) input
images. In contrast to existing methods that often learn a single
patch-to-patch mapping from LR to HR images and are regardless of the
contextual interdependency between patches, we propose a novel Attention-aware
Face Hallucination (Attention-FH) framework which resorts to deep reinforcement
learning for sequentially discovering attended patches and then performing the
facial part enhancement by fully exploiting the global interdependency of the
image. Specifically, in each time step, the recurrent policy network is
proposed to dynamically specify a new attended region by incorporating what
happened in the past. The state (i.e., face hallucination result for the whole
image) can thus be exploited and updated by the local enhancement network on
the selected region. The Attention-FH approach jointly learns the recurrent
policy network and local enhancement network through maximizing the long-term
reward that reflects the hallucination performance over the whole image.
Therefore, our proposed Attention-FH is capable of adaptively personalizing an
optimal searching path for each face image according to its own characteristic.
Extensive experiments show our approach significantly surpasses the
state-of-the-arts on in-the-wild faces with large pose and illumination
variations
Deep Learning for Semantic Part Segmentation with High-Level Guidance
In this work we address the task of segmenting an object into its parts, or
semantic part segmentation. We start by adapting a state-of-the-art semantic
segmentation system to this task, and show that a combination of a
fully-convolutional Deep CNN system coupled with Dense CRF labelling provides
excellent results for a broad range of object categories. Still, this approach
remains agnostic to high-level constraints between object parts. We introduce
such prior information by means of the Restricted Boltzmann Machine, adapted to
our task and train our model in an discriminative fashion, as a hidden CRF,
demonstrating that prior information can yield additional improvements. We also
investigate the performance of our approach ``in the wild'', without
information concerning the objects' bounding boxes, using an object detector to
guide a multi-scale segmentation scheme. We evaluate the performance of our
approach on the Penn-Fudan and LFW datasets for the tasks of pedestrian parsing
and face labelling respectively. We show superior performance with respect to
competitive methods that have been extensively engineered on these benchmarks,
as well as realistic qualitative results on part segmentation, even for
occluded or deformable objects. We also provide quantitative and extensive
qualitative results on three classes from the PASCAL Parts dataset. Finally, we
show that our multi-scale segmentation scheme can boost accuracy, recovering
segmentations for finer parts.Comment: 11 pages (including references), 3 figures, 2 table
Some like it hot - visual guidance for preference prediction
For people first impressions of someone are of determining importance. They
are hard to alter through further information. This begs the question if a
computer can reach the same judgement. Earlier research has already pointed out
that age, gender, and average attractiveness can be estimated with reasonable
precision. We improve the state-of-the-art, but also predict - based on
someone's known preferences - how much that particular person is attracted to a
novel face. Our computational pipeline comprises a face detector, convolutional
neural networks for the extraction of deep features, standard support vector
regression for gender, age and facial beauty, and - as the main novelties -
visual regularized collaborative filtering to infer inter-person preferences as
well as a novel regression technique for handling visual queries without rating
history. We validate the method using a very large dataset from a dating site
as well as images from celebrities. Our experiments yield convincing results,
i.e. we predict 76% of the ratings correctly solely based on an image, and
reveal some sociologically relevant conclusions. We also validate our
collaborative filtering solution on the standard MovieLens rating dataset,
augmented with movie posters, to predict an individual's movie rating. We
demonstrate our algorithms on howhot.io which went viral around the Internet
with more than 50 million pictures evaluated in the first month.Comment: accepted for publication at CVPR 201
- …