We present a modular framework for articulatory animation synthesis using
speech motion capture data obtained with electromagnetic articulography (EMA).
Adapting a skeletal animation approach, the articulatory motion data is applied
to a three-dimensional (3D) model of the vocal tract, creating a portable
resource that can be integrated in an audiovisual (AV) speech synthesis
platform to provide realistic animation of the tongue and teeth for a virtual
character. The framework also provides an interface to articulatory animation
synthesis, as well as an example application to illustrate its use with a 3D
game engine. We rely on cross-platform, open-source software and open standards
to provide a lightweight, accessible, and portable workflow.Comment: Workshop on Innovation and Applications in Speech Technology (2012

Ouni, Slim

Steiner, Ingmar

English

arXiv

We present a modular framework for articulatory animation synthesis using speech motion capture data obtained with elec-tromagnetic articulography (EMA). Adapting a skeletal anima-tion approach, the articulatory motion data is applied to a three-dimensional (3D) model of the vocal tract, creating a portable resource that can be integrated in an audiovisual (AV) speech synthesis platform to provide realistic animation of the tongue and teeth for a virtual character. The framework also provides an interface to articulatory animation synthesis, as well as an example application to illustrate its use with a 3D game engine. We rely on cross-platform, open-source software and open stan-dards to provide a lightweight, accessible, and portable work-flow

Ingmar Steiner

Slim Ouni

CiteSeerX

Artimate: an articulatory animation framework for audiovisual speech synthesis

International audienceWe present a modular framework for articulatory animation synthesis using speech motion capture data obtained with electromagnetic articulography (EMA). Adapting a skeletal animation approach, the articulatory motion data is applied to a three-dimensional (3D) model of the vocal tract, creating a portable resource that can be integrated in an audiovisual (AV) speech synthesis platform to provide realistic animation of the tongue and teeth for a virtual character. The framework also provides an interface to articulatory animation synthesis, as well as an example application to illustrate its use with a 3D game engine. We rely on cross-platform, open-source software and open standards to provide a lightweight, accessible, and portable workflow

INRIA a CCSD electronic archive server

Introduction Modelling Architecture ConclusionArtimate: an articulatory animation frameworkfor audiovisual speech synthesisIngmar Steiner and Slim OuniSpeech GroupDublin, March 9, 2012Introduction Modelling Architecture ConclusionMotion CaptureFace/Body Performancesource:photobucket.comsource:tumblr.comIntroduction Modelling Architecture ConclusionMotion CaptureFace/Body Performancesource:photobucket.comsource:tumblr.comIntroduction Modelling Architecture ConclusionMotion CaptureFace/Body Performancesource:photobucket.comsource:tumblr.comIntroduction Modelling Architecture ConclusionMotion CaptureSpeech?source:turbosquid.comIntroduction Modelling Architecture ConclusionMotion CaptureSpeech?source:turbosquid.comIntroduction Modelling Architecture ConclusionMotion CaptureSpeech?source:turbosquid.comIntroduction Modelling Architecture ConclusionModel AssemblyStatic model & EMAIntroduction Modelling Architecture ConclusionModel AssemblyRiggingIntroduction Modelling Architecture ConclusionModel AnimationSkeletonIntroduction Modelling Architecture ConclusionModel AnimationSurfaceIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionArchitectureArtimate Model CompilerArtimate Core ... .. ..M .A.R .Y .. . .. . .. . .. . ..ext.. . .. . .. . .. . ..o .. . .. . .. . .. ..peechArtimate Demo AppvenmaIntroduction Modelling Architecture ConclusionSummary• Adapted skeletal animation for articulatory animation• Motion capture data (EMA)• Modular, cross-platform implementation with open-sourcedependencies• Self-contained, portable resourceIntroduction Modelling Architecture ConclusionSummary• Adapted skeletal animation for articulatory animation• Motion capture data (EMA)• Modular, cross-platform implementation with open-sourcedependencies• Self-contained, portable resourceIntroduction Modelling Architecture ConclusionSummary• Adapted skeletal animation for articulatory animation• Motion capture data (EMA)• Modular, cross-platform implementation with open-sourcedependencies• Self-contained, portable resourceIntroduction Modelling Architecture ConclusionSummary• Adapted skeletal animation for articulatory animation• Motion capture data (EMA)• Modular, cross-platform implementation with open-sourcedependencies• Self-contained, portable resourceIntroduction Modelling Architecture ConclusionFurther work• Handle noisy EMA data• Testing (validation, portability, integration, . . . )• Documentation• Public releaseIntroduction Modelling Architecture ConclusionFurther work• Handle noisy EMA data• Testing (validation, portability, integration, . . . )• Documentation• Public releaseIntroduction Modelling Architecture ConclusionFurther work• Handle noisy EMA data• Testing (validation, portability, integration, . . . )• Documentation• Public releaseIntroduction Modelling Architecture ConclusionFurther work• Handle noisy EMA data• Testing (validation, portability, integration, . . . )• Documentation• Public release

https://hal.inria.fr/hal-00678964/file/slides.pdf

Artimate: an articulatory animation framework for audiovisual speech synthesis

Abstract

Similar works

Full text

Available Versions

CiteSeerX

INRIA a CCSD electronic archive server

CiteSeerX