19 research outputs found
Deep representation learning for human motion prediction and classification
Generative models of 3D human motion are often restricted to a small number
of activities and can therefore not generalize well to novel movements or
applications. In this work we propose a deep learning framework for human
motion capture data that learns a generic representation from a large corpus of
motion capture data and generalizes well to new, unseen, motions. Using an
encoding-decoding network that learns to predict future 3D poses from the most
recent past, we extract a feature representation of human motion. Most work on
deep learning for sequence prediction focuses on video and speech. Since
skeletal data has a different structure, we present and evaluate different
network architectures that make different assumptions about time dependencies
and limb correlations. To quantify the learned features, we use the output of
different layers for action classification and visualize the receptive fields
of the network units. Our method outperforms the recent state of the art in
skeletal motion prediction even though these use action specific training data.
Our results show that deep feedforward networks, trained from a generic mocap
database, can successfully be used for feature extraction from human motion
data and that this representation can be used as a foundation for
classification and prediction.Comment: This paper is published at the IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), 201
Analyzing Input and Output Representations for Speech-Driven Gesture Generation
This paper presents a novel framework for automatic speech-driven gesture
generation, applicable to human-agent interaction including both virtual agents
and robots. Specifically, we extend recent deep-learning-based, data-driven
methods for speech-driven gesture generation by incorporating representation
learning. Our model takes speech as input and produces gestures as output, in
the form of a sequence of 3D coordinates. Our approach consists of two steps.
First, we learn a lower-dimensional representation of human motion using a
denoising autoencoder neural network, consisting of a motion encoder MotionE
and a motion decoder MotionD. The learned representation preserves the most
important aspects of the human pose variation while removing less relevant
variation. Second, we train a novel encoder network SpeechE to map from speech
to a corresponding motion representation with reduced dimensionality. At test
time, the speech encoder and the motion decoder networks are combined: SpeechE
predicts motion representations based on a given speech signal and MotionD then
decodes these representations to produce motion sequences. We evaluate
different representation sizes in order to find the most effective
dimensionality for the representation. We also evaluate the effects of using
different speech features as input to the model. We find that mel-frequency
cepstral coefficients (MFCCs), alone or combined with prosodic features,
perform the best. The results of a subsequent user study confirm the benefits
of the representation learning.Comment: Accepted at IVA '19. Shorter version published at AAMAS '19. The code
is available at
https://github.com/GestureGeneration/Speech_driven_gesture_generation_with_autoencode
Filter-adapted spatiotemporal sampling for real-time rendering
Stochastic sampling techniques are ubiquitous in real-time rendering, where
performance constraints force the use of low sample counts, leading to noisy
intermediate results. To remove this noise, the post-processing step of
temporal and spatial denoising is an integral part of the real-time graphics
pipeline. The main insight presented in this paper is that we can optimize the
samples used in stochastic sampling such that the post-processing error is
minimized. The core of our method is an analytical loss function which measures
post-filtering error for a class of integrands - multidimensional Heaviside
functions. These integrands are an approximation of the discontinuous functions
commonly found in rendering. Our analysis applies to arbitrary spatial and
spatiotemporal filters, scalar and vector sample values, and uniform and
non-uniform probability distributions. We show that the spectrum of Monte Carlo
noise resulting from our sampling method is adapted to the shape of the filter,
resulting in less noisy final images. We demonstrate improvements over
state-of-the-art sampling methods in three representative rendering tasks:
ambient occlusion, volumetric ray-marching, and color image dithering. Common
use noise textures, and noise generation code is available at
https://github.com/electronicarts/fastnoise.Comment: 18 pages, 12 figure
Sociala sensorimotoriska funktioner
As the field of robotics advances, more robots are employed in our everyday environment. Thus, the implementation of robots that can actively engage in physical collaboration and naturally interact with humans is of high importance. In order to achieve this goal, it is necessary to study human interaction and social cognition and how these aspects can be implemented in robotic agents. The theory of social sensorimotor contingencies hypothesises that many aspects of human-human interaction depend on low-level signalling and mutual prediction. In this thesis, I give an extensive account of these underlying mechanisms and how research in human-robot interaction has incorporated this knowledge. I integrate this work in human-human and human-robot interaction into a coherent framework of social sensorimotor contingencies. Furthermore, I devise a generative model based on low-level latent features that allows inferences about other agent's behaviour. With this simulation experiment I demonstrate that embodied cognition can explain behaviour that is usually interpreted with help of high-level belief and mental state inferences. In conclusion, the implementation of these low-level processes in robots creates a more natural and intuitive interaction without the need of high-level representations