Artimate: an articulatory animation framework for audiovisual speech synthesis

We present a modular framework for articulatory animation synthesis using speech motion capture data obtained with electromagnetic articulography (EMA). Adapting a skeletal animation approach, the articulatory motion data is applied to a three-dimensional (3D) model of the vocal tract, creating a portable resource that can be integrated in an audiovisual (AV) speech synthesis platform to provide realistic animation of the tongue and teeth for a virtual character. The framework also provides an interface to articulatory animation synthesis, as well as an example application to illustrate its use with a 3D game engine. We rely on cross-platform, open-source software and open standards to provide a lightweight, accessible, and portable workflow.Comment: Workshop on Innovation and Applications in Speech Technology (2012

A biomechanical modeling study of the effects of the orbicularis oris muscle and jaw posture on lip shape

Purpose: The authors' general aim is to use biomechanical models of speech articulators to explore how possible variations in anatomical structure contribute to differences in articulatory strategies and phone systems across human populations. Specifically, they investigated 2 issues: (a) the link between lip muscle anatomy and variability in lip gestures and (b) the constraints of coupled lip/jaw biomechanics on jaw posture in labial sounds. Method: The authors used a model coupling the jaw, tongue, and face. First, the influence of the orbicularis oris (OO) anatomical implementation was analyzed by assessing how changes in depth (from epidermis to the skull) and peripheralness (proximity to the lip horn center) affected lip shaping. Second, the capability of the lip/jaw system to generate protrusion and rounding, or labial closure, was evaluated for different jaw heights. Results: Results showed that a peripheral and moderately deep OO implementation is most appropriate for protrusion and rounding; a superficial implementation facilitates closure; protrusion and rounding require a high jaw position; and closure is achievable for various jaw heights. Conclusions: Models provide objective information regarding possible links between anatomical and speech production variability across humans. Comparisons with experimental data will illustrate how motor control and cultural factors cope with these constraints

The Distributed Lambda Model (DLM): A 3-D, Finite-Element Muscle Model Based on Feldman's Lambda Model; Assessment of Orofacial Gestures

International audiencePurpose: The authors aimed to design a distributed Lambda model (DLM), which is well-adapted to implement three-dimensional (3-D) Finite Element descriptions of muscles. Method: A muscle element model was designed. Its stress-strain relationships included the active force-length characteristics of the Lambda model along the muscle fibers, together with the passive properties of muscle tissues in the 3-D space. The muscle element was first assessed using simple geometrical representations of muscles in form of rectangular bars. Then, it was included in a 3-D face model, and its impact on lip protrusion was compared with the impact of a Hill-type muscle model. Results: The force-length characteristic associated with the muscle elements matched well with the invariant characteristics of the Lambda model. The impact of the passive properties was assessed. Isometric force variation and isotonic displacements were modeled. The comparison with a Hill-type model revealed strong similarities in terms of global stress and strain. Conclusion: The DLM accounted for the characteristics of the Lambda model. Biomechanically no clear differences were found between the DLM and a Hill-type model. Accurate evaluations of the Lambda model, based on the comparison between data and simulations, are now possible with 3-D biomechanical descriptions of the speech articulators because to the DLM

Biomechanical Models of Human Upper and Tracheal Airway Functionality

The respiratory tract, in other words, the airway, is the primary airflow path for several physiological activities such as coughing, breathing, and sneezing. Diseases can impact airway functionality through various means including cancer of the head and neck, Neurological disorders such as Parkinson\u27s disease, and sleep disorders and all of which are considered in this study. In this dissertation, numerical modeling techniques were used to simulate three distinct airway diseases: a weak cough leading to aspiration, upper airway patency in obstructive sleep apnea, and tongue cancer in swallow disorders. The work described in this dissertation, therefore, divided into three biomechanical models, of which fluid and particulate dynamics model of cough is the first. Cough is an airway protective mechanism, which results from a coordinated series of respiratory, laryngeal, and pharyngeal muscle activity. Patients with diminished upper airway protection often exhibit cough impairment resulting in aspiration pneumonia. Computational Fluid Dynamics (CFD) technique was used to simulate airflow and penetrant behavior in the airway geometry reconstructed from Computed Tomography (CT) images acquired from participants. The second study describes Obstructive Sleep Apnea (OSA) and the effects of dilator muscular activation on the human retro-lingual airway in OSA. Computations were performed for the inspiration stage of the breathing cycle, utilizing a fluid-structure interaction (FSI) method to couple structural deformation with airflow dynamics. The spatiotemporal deformation of the structures surrounding the airway wall was predicted and found to be in general agreement with observed changes in luminal opening and the distribution of airflow from upright to supine posture. The third study describes the effects of cancer of the tongue base on tongue motion during swallow. A three-dimensional biomechanical model was developed and used to calculate the spatiotemporal deformation of the tongue under a sequence of movements which simulate the oral stage of swallow

Three speech sounds, one motor action: Evidence for speech-motor disparity from English flap production

The assumption that units of speech production bear a one-to-one relationship to speech motor actions pervades otherwise widely varying theories of speech motor behavior. This speech produc- tion and simulation study demonstrates that commonly occurring flap sequences may violate this assumption. In the word “Saturday,” a sequence of three sounds may be produced using a single, cyclic motor action. Under this view, the initial upward tongue tip motion, starting with the first vowel and moving to contact the hard palate on the way to a retroflex position, is under active muscular control, while the downward movement of the tongue tip, including the second contact with the hard palate, results from gravity and elasticity during tongue muscle relaxation. This sequence is reproduced using a three-dimensional computer simulation of human vocal tract biomechanics and differs greatly from other observed sequences for the same word, which employ multiple targeted speech motor actions. This outcome suggests that a goal of a speaker is to produce an entire sequence in a biomechanically efficient way at the expense of maintaining parity within the individual parts of the sequence

A computational neuromuscular model of the human upper airway with application to the study of obstructive sleep apnoea

Includes bibliographical references.Numerous challenges are faced in investigations aimed at developing a better understanding of the pathophysiology of obstructive sleep apnoea. The anatomy of the tongue and other upper airway tissues, and the ability to model their behaviour, is central to such investigations. In this thesis, details of the construction and development of a three-dimensional finite element model of soft tissues of the human upper airway, as well as a simplified fluid model of the airway, are provided. The anatomical data was obtained from the Visible Human Project, and its underlying micro-histological data describing tongue musculature were also extracted from the same source and incorporated into the model. An overview of the mathematical models used to describe tissue behaviour, both at a macro- and microscopic level, is given. Hyperelastic constitutive models were used to describe the material behaviour, and material incompressibility was accounted for. An active Hill three-element muscle model was used to represent the muscular tissue of the tongue. The neural stimulus for each muscle group to a priori unknown external forces was determined through the use of a genetic algorithm-based neural control model. The fundamental behaviour of the tongue under gravitational and breathing-induced loading is investigated. The response of the various muscles of the tongue to the complex loading developed during breathing is determined, with a particular focus being placed to that of the genioglossus. It is demonstrated that, when a time-dependent loading is applied to the tongue, the neural model is able to control the position of the tongue and produce a physiologically realistic response for the genioglossus. A comparison is then made to the response determined under quasi-static conditions using the pressure distribution extracted from computational fluid-dynamics results. An analytical model describing the time-dependent response of the components of the tongue musculature most active during oral breathing is developed and validated. It is then modified to simulate the activity of the tongue during sleep and under conditions relating to various possible neural and physiological pathologies. The retroglossal movement of the tongue resulting from the pathologies is quantified and their role in the potential to induce airway collapse is discussed

Face

The face is probably the part of the body, which most distinguishes us as individuals. It plays a very important role in many functions, such as speech, mastication, and expression of emotion. In the face, there is a tight coupling between different complex structures, such as skin, fat, muscle, and bone. Biomechanically driven models of the face provide an opportunity to gain insight into how these different facial components interact. The benefits of this insight are manifold, including improved maxillofacial surgical planning, better understanding of speech mechanics, and more realistic facial animations. This chapter provides an overview of facial anatomy followed by a review of previous computational models of the face. These models include facial tissue constitutive relationships, facial muscle models, and finite element models. We also detail our efforts to develop novel general and subject-specific models. We present key results from simulations that highlight the realism of the face models