3,202 research outputs found
Learning Human Motion Models for Long-term Predictions
We propose a new architecture for the learning of predictive spatio-temporal
motion models from data alone. Our approach, dubbed the Dropout Autoencoder
LSTM, is capable of synthesizing natural looking motion sequences over long
time horizons without catastrophic drift or motion degradation. The model
consists of two components, a 3-layer recurrent neural network to model
temporal aspects and a novel auto-encoder that is trained to implicitly recover
the spatial structure of the human skeleton via randomly removing information
about joints during training time. This Dropout Autoencoder (D-AE) is then used
to filter each predicted pose of the LSTM, reducing accumulation of error and
hence drift over time. Furthermore, we propose new evaluation protocols to
assess the quality of synthetic motion sequences even for which no ground truth
data exists. The proposed protocols can be used to assess generated sequences
of arbitrary length. Finally, we evaluate our proposed method on two of the
largest motion-capture datasets available to date and show that our model
outperforms the state-of-the-art on a variety of actions, including cyclic and
acyclic motion, and that it can produce natural looking sequences over longer
time horizons than previous methods
An original framework for understanding human actions and body language by using deep neural networks
The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour.
By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way.
These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively.
While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements;
both are essential tasks in many computer vision applications, including event recognition, and video surveillance.
In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided.
The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements.
All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods
Action Capsules: Human Skeleton Action Recognition
Due to the compact and rich high-level representations offered,
skeleton-based human action recognition has recently become a highly active
research topic. Previous studies have demonstrated that investigating joint
relationships in spatial and temporal dimensions provides effective information
critical to action recognition. However, effectively encoding global
dependencies of joints during spatio-temporal feature extraction is still
challenging. In this paper, we introduce Action Capsule which identifies
action-related key joints by considering the latent correlation of joints in a
skeleton sequence. We show that, during inference, our end-to-end network pays
attention to a set of joints specific to each action, whose encoded
spatio-temporal features are aggregated to recognize the action. Additionally,
the use of multiple stages of action capsules enhances the ability of the
network to classify similar actions. Consequently, our network outperforms the
state-of-the-art approaches on the N-UCLA dataset and obtains competitive
results on the NTURGBD dataset. This is while our approach has significantly
lower computational requirements based on GFLOPs measurements.Comment: 11 pages, 11 figure
Skeleton-based Relational Reasoning for Group Activity Analysis
Research on group activity recognition mostly leans on the standard
two-stream approach (RGB and Optical Flow) as their input features. Few have
explored explicit pose information, with none using it directly to reason about
the persons interactions. In this paper, we leverage the skeleton information
to learn the interactions between the individuals straight from it. With our
proposed method GIRN, multiple relationship types are inferred from independent
modules, that describe the relations between the body joints pair-by-pair.
Additionally to the joints relations, we also experiment with the previously
unexplored relationship between individuals and relevant objects (e.g.
volleyball). The individuals distinct relations are then merged through an
attention mechanism, that gives more importance to those individuals more
relevant for distinguishing the group activity. We evaluate our method in the
Volleyball dataset, obtaining competitive results to the state-of-the-art. Our
experiments demonstrate the potential of skeleton-based approaches for modeling
multi-person interactions.Comment: 26 pages, 5 figures, accepted manuscript in Elsevier Pattern
Recognition, minor writing revisions and new reference
- …