1,040 research outputs found
Comparison of HMM and TMDN Methods for Lip Synchronisation
This paper presents a comparison between a hidden Markov model (HMM) based method and a novel artificial neural network (ANN) based method for lip synchronisation. Both model types were trained on motion tracking data and a perceptual evaluation was carried out comparing the output of the models, both to each other and to the original tracked data. It was found that the ANN based method was judged significantly better than the HMM based method. Furthermore the original data was not judged significantly better than the output of the ANN method. Index Terms: hidden Markov model, mixture density network, lip synchronisation, inversion mappin
Comparison of HMM and TMD Methods for Lip Synchronisation
This paper presents a comparison between a hidden Markov model (HMM) based method and a novel artificial neural network (ANN) based method for lip synchronisation. Both model types were trained on motion tracking data, and a perceptual evaluation was carried out comparing the output of the models, both to each other and to the original tracked data. It was found that the ANN-based method was judged significantly better than the HMM based method. Furthermore, the original data was not judged significantly better than the output of the ANN method
Speech-driven animation using multi-modal hidden Markov models
The main objective of this thesis was the synthesis of speech synchronised motion, in
particular head motion. The hypothesis that head motion can be estimated from the
speech signal was confirmed. In order to achieve satisfactory results, a motion capture
data base was recorded, a definition of head motion in terms of articulation was discovered,
a continuous stream mapping procedure was developed, and finally the synthesis
was evaluated. Based on previous research into non-verbal behaviour basic types of
head motion were invented that could function as modelling units. The stream mapping
method investigated in this thesis is based on Hidden Markov Models (HMMs), which
employ modelling units to map between continuous signals. The objective evaluation
of the modelling parameters confirmed that head motion types could be predicted from
the speech signal with an accuracy above chance, close to 70%. Furthermore, a special
type ofHMMcalled trajectoryHMMwas used because it enables synthesis of continuous
output. However head motion is a stochastic process therefore the trajectory HMM
was further extended to allow for non-deterministic output. Finally the resulting head
motion synthesis was perceptually evaluated. The effects of the âuncanny valleyâ were
also considered in the evaluation, confirming that rendering quality has an influence on
our judgement of movement of virtual characters. In conclusion a general method for
synthesising speech-synchronised behaviour was invented that can applied to a whole
range of behaviours
Automatic Head Motion Prediction from Speech Data
In this paper we present a novel approach to generate a
sequence of head motion units given some speech. The
modelling approach is based on the notion that head motion
can be divided into a number of short homogeneous units that
can be modelled individually. The system is based on Hidden
Markov Models (HMM), which are trained on motion units
and act as a sequence generator. They can be evaluated by
an accuracy measure. A database of motion capture data was
collected and manually annotated for head motion and is used
to train the models. It was found that the model is good at
distinguishing high activity regions from regions with less
activity with accuracies around 75 percent. Furthermore the
model is able to distinguish different head motion patterns
based on speech features somewhat reliably, with accuracies
reaching almost 70 percent
Speech-driven head motion synthesis based on a trajectory model.
Making human-like characters more natural and life-like requires
more inventive approaches than current standard techniques such
as synthesis using text features or triggers. In this poster we present
a novel approach to automatically synthesise head motion based on
speech features. Previous work has focused on frame wise modelling
of motion [Busso et al. 2007] or has treated the speach data
and motion data streams separately [Brand 1999], although the trajectories
of the head motion and speech features are highly correlated
and dynamically change over several frames. To model
longer units of motion and speech and to reproduce their trajectories
during synthesis, we utilise a promising time series stochastic
model called âTrajectory Hidden Markov Modelsâ [Zen et al.
2007]. Its parameter generation algorithm can produce motion trajectories
from sequences of units of motion and speech. These
two kinds of data are simultaneously modelled by using a multistream
type of the trajectory HMMs. The models can be viewed
as a Kalman-smoother-like approach, and thereby are capable of
producing smooth trajectories
Lip motion synthesis using a context dependent trajectory hidden Markov model
Lip synchronisation is essential to make character animation believeable. In this poster we present a novel technique
to automatically synthesise lip motion trajectories given some text and speech. Our work distinguishes itself
from other work by not using visemes (visual counterparts of phonemes). The lip motion trajectories are directly
modelled using a time series stochastic model called âTrajectory Hidden Markov Modelâ. Its parameter generation
algorithm can produce motion trajectories that are used to drive control points on the lips directly
- âŚ