3,890 research outputs found
Multi-View Face Recognition From Single RGBD Models of the Faces
This work takes important steps towards solving the following problem of current interest: Assuming that each individual in a population can be modeled by a single frontal RGBD face image, is it possible to carry out face recognition for such a population using multiple 2D images captured from arbitrary viewpoints? Although the general problem as stated above is extremely challenging, it encompasses subproblems that can be addressed today. The subproblems addressed in this work relate to: (1) Generating a large set of viewpoint dependent face images from a single RGBD frontal image for each individual; (2) using hierarchical approaches based on view-partitioned subspaces to represent the training data; and (3) based on these hierarchical approaches, using a weighted voting algorithm to integrate the evidence collected from multiple images of the same face as recorded from different viewpoints. We evaluate our methods on three datasets: a dataset of 10 people that we created and two publicly available datasets which include a total of 48 people. In addition to providing important insights into the nature of this problem, our results show that we are able to successfully recognize faces with accuracies of 95% or higher, outperforming existing state-of-the-art face recognition approaches based on deep convolutional neural networks
Robust Estimation of 3D Human Poses from a Single Image
Human pose estimation is a key step to action recognition. We propose a
method of estimating 3D human poses from a single image, which works in
conjunction with an existing 2D pose/joint detector. 3D pose estimation is
challenging because multiple 3D poses may correspond to the same 2D pose after
projection due to the lack of depth information. Moreover, current 2D pose
estimators are usually inaccurate which may cause errors in the 3D estimation.
We address the challenges in three ways: (i) We represent a 3D pose as a linear
combination of a sparse set of bases learned from 3D human skeletons. (ii) We
enforce limb length constraints to eliminate anthropomorphically implausible
skeletons. (iii) We estimate a 3D pose by minimizing the -norm error
between the projection of the 3D pose and the corresponding 2D detection. The
-norm loss term is robust to inaccurate 2D joint estimations. We use the
alternating direction method (ADM) to solve the optimization problem
efficiently. Our approach outperforms the state-of-the-arts on three benchmark
datasets
Enabling viewpoint learning through dynamic label generation
Optimal viewpoint prediction is an essential task in many computer graphics applications. Unfortunately, common viewpointqualities suffer from two major drawbacks: dependency on clean surface meshes, which are not always available, and the lack ofclosed-form expressions, which requires a costly search involving rendering. To overcome these limitations we propose to sepa-rate viewpoint selection from rendering through an end-to-end learning approach, whereby we reduce the in¿uence of the meshquality by predicting viewpoints from unstructured point clouds instead of polygonal meshes. While this makes our approachinsensitive to the mesh discretization during evaluation, it only becomes possible when resolving label ambiguities that arise inthis context. Therefore, we additionally propose to incorporate the label generation into the training procedure, making the labeldecision adaptive to the current network predictions. We show how our proposed approach allows for learning viewpoint pre-dictions for models from different object categories and for different viewpoint qualities. Additionally, we show that predictiontimes are reduced from several minutes to a fraction of a second, as compared to state-of-the-art (SOTA) viewpoint quality eval-uation. Code and training data is available at https://github.com/schellmi42/viewpoint_learning, whichis to our knowledge the biggest viewpoint quality dataset available.This work was supported in part by project TIN2017-88515-C2-1-R(GEN3DLIVE), from the Spanish Ministerio de Economía yCompetitividad, by 839 FEDER (EU) funds.Peer ReviewedPostprint (published version
Advances in Monocular Exemplar-based Human Body Pose Analysis: Modeling, Detection and Tracking
Esta tesis contribuye en el análisis de la postura del cuerpo humano a partir de secuencias de imágenes adquiridas con una sola cámara. Esta temática presenta un amplio rango de potenciales aplicaciones en video-vigilancia, video-juegos o aplicaciones biomédicas. Las técnicas basadas en patrones han tenido éxito, sin embargo, su precisión depende de la similitud del punto de vista de la cámara y de las propiedades de la escena entre las imágenes de entrenamiento y las de prueba. Teniendo en cuenta un conjunto de datos de entrenamiento capturado mediante un número reducido de cámaras fijas, paralelas al suelo, se han identificado y analizado tres escenarios posibles con creciente nivel de dificultad: 1) una cámara estática paralela al suelo, 2) una cámara de vigilancia fija con un ángulo de visión considerablemente diferente, y 3) una secuencia de video capturada con una cámara en movimiento o simplemente una sola imagen estática
Graph-based classification of multiple observation sets
We consider the problem of classification of an object given multiple
observations that possibly include different transformations. The possible
transformations of the object generally span a low-dimensional manifold in the
original signal space. We propose to take advantage of this manifold structure
for the effective classification of the object represented by the observation
set. In particular, we design a low complexity solution that is able to exploit
the properties of the data manifolds with a graph-based algorithm. Hence, we
formulate the computation of the unknown label matrix as a smoothing process on
the manifold under the constraint that all observations represent an object of
one single class. It results into a discrete optimization problem, which can be
solved by an efficient and low complexity algorithm. We demonstrate the
performance of the proposed graph-based algorithm in the classification of sets
of multiple images. Moreover, we show its high potential in video-based face
recognition, where it outperforms state-of-the-art solutions that fall short of
exploiting the manifold structure of the face image data sets.Comment: New content adde
REPRESENTATION LEARNING FOR ACTION RECOGNITION
The objective of this research work is to develop discriminative representations for human
actions. The motivation stems from the fact that there are many issues encountered while
capturing actions in videos like intra-action variations (due to actors, viewpoints, and duration),
inter-action similarity, background motion, and occlusion of actors. Hence, obtaining
a representation which can address all the variations in the same action while maintaining
discrimination with other actions is a challenging task. In literature, actions have been represented
either using either low-level or high-level features. Low-level features describe
the motion and appearance in small spatio-temporal volumes extracted from a video. Due
to the limited space-time volume used for extracting low-level features, they are not able
to account for viewpoint and actor variations or variable length actions. On the other hand,
high-level features handle variations in actors, viewpoints, and duration but the resulting
representation is often high-dimensional which introduces the curse of dimensionality. In
this thesis, we propose new representations for describing actions by combining the advantages
of both low-level and high-level features. Specifically, we investigate various linear
and non-linear decomposition techniques to extract meaningful attributes in both high-level
and low-level features. In the first approach, the sparsity of high-level feature descriptors is leveraged to build
action-specific dictionaries. Each dictionary retains only the discriminative information
for a particular action and hence reduces inter-action similarity. Then, a sparsity-based
classification method is proposed to classify the low-rank representation of clips obtained
using these dictionaries. We show that this representation based on dictionary learning improves
the classification performance across actions. Also, a few of the actions consist of
rapid body deformations that hinder the extraction of local features from body movements.
Hence, we propose to use a dictionary which is trained on convolutional neural network
(CNN) features of the human body in various poses to reliably identify actors from the
background. Particularly, we demonstrate the efficacy of sparse representation in the identification
of the human body under rapid and substantial deformation.
In the first two approaches, sparsity-based representation is developed to improve discriminability
using class-specific dictionaries that utilize action labels. However, developing
an unsupervised representation of actions is more beneficial as it can be used to both
recognize similar actions and localize actions. We propose to exploit inter-action similarity
to train a universal attribute model (UAM) in order to learn action attributes (common and
distinct) implicitly across all the actions. Using maximum aposteriori (MAP) adaptation,
a high-dimensional super action-vector (SAV) for each clip is extracted. As this SAV contains
redundant attributes of all other actions, we use factor analysis to extract a novel lowvi
dimensional action-vector representation for each clip. Action-vectors are shown to suppress
background motion and highlight actions of interest in both trimmed and untrimmed
clips that contributes to action recognition without the help of any classifiers.
It is observed during our experiments that action-vector cannot effectively discriminate
between actions which are visually similar to each other. Hence, we subject action-vectors
to supervised linear embedding using linear discriminant analysis (LDA) and probabilistic
LDA (PLDA) to enforce discrimination. Particularly, we show that leveraging complimentary
information across action-vectors using different local features followed by discriminative
embedding provides the best classification performance. Further, we explore
non-linear embedding of action-vectors using Siamese networks especially for fine-grained
action recognition. A visualization of the hidden layer output in Siamese networks shows
its ability to effectively separate visually similar actions. This leads to better classification
performance than linear embedding on fine-grained action recognition.
All of the above approaches are presented on large unconstrained datasets with hundreds
of examples per action. However, actions in surveillance videos like snatch thefts are
difficult to model because of the diverse variety of scenarios in which they occur and very
few labeled examples. Hence, we propose to utilize the universal attribute model (UAM)
trained on large action datasets to represent such actions. Specifically, we show that there
are similarities between certain actions in the large datasets with snatch thefts which help
in extracting a representation for snatch thefts using the attributes from the UAM. This
representation is shown to be effective in distinguishing snatch thefts from regular actions
with high accuracy.In summary, this thesis proposes both supervised and unsupervised approaches for representing
actions which provide better discrimination than existing representations. The
first approach presents a dictionary learning based sparse representation for effective discrimination
of actions. Also, we propose a sparse representation for the human body based
on dictionaries in order to recognize actions with rapid body deformations. In the next
approach, a low-dimensional representation called action-vector for unsupervised action
recognition is presented. Further, linear and non-linear embedding of action-vectors is
proposed for addressing inter-action similarity and fine-grained action recognition, respectively.
Finally, we propose a representation for locating snatch thefts among thousands of
regular interactions in surveillance videos
- …