THREE DIMENSIONAL MODELING AND ANIMATION OF FACIAL EXPRESSIONS

Facial expression and animation are important aspects of the 3D environment featuring human characters. These animations are frequently used in many kinds of applications and there have been many efforts to increase the realism. Three aspects are still stimulating active research: the detailed subtle facial expressions, the process of rigging a face, and the transfer of an expression from one person to another. This dissertation focuses on the above three aspects. A system for freely designing and creating detailed, dynamic, and animated facial expressions is developed. The presented pattern functions produce detailed and animated facial expressions. The system produces realistic results with fast performance, and allows users to directly manipulate it and see immediate results. Two unique methods for generating real-time, vivid, and animated tears have been developed and implemented. One method is for generating a teardrop that continually changes its shape as the tear drips down the face. The other is for generating a shedding tear, which is a kind of tear that seamlessly connects with the skin as it flows along the surface of the face, but remains an individual object. The methods both broaden CG and increase the realism of facial expressions. A new method to automatically set the bones on facial/head models to speed up the rigging process of a human face is also developed. To accomplish this, vertices that describe the face/head as well as relationships between each part of the face/head are grouped. The average distance between pairs of vertices is used to place the head bones. To set the bones in the face with multi-density, the mean value of the vertices in a group is measured. The time saved with this method is significant. A novel method to produce realistic expressions and animations by transferring an existing expression to a new facial model is developed. The approach is to transform the source model into the target model, which then has the same topology as the source model. The displacement vectors are calculated. Each vertex in the source model is mapped to the target model. The spatial relationships of each mapped vertex are constrained

Example Based Image Analysis and Synthesis

Image analysis and graphics synthesis can be achieved with learning techniques using directly image examples without physically-based, 3D models. In our technique: -- the mapping from novel images to a vector of "pose" and "expression" parameters can be learned from a small set of example images using a function approximation technique that we call an analysis network; -- the inverse mapping from input "pose" and "expression" parameters to output images can be synthesized from a small set of example images and used to produce new images using a similar synthesis network. The techniques described here have several applications in computer graphics, special effects, interactive multimedia and very low bandwidth teleconferencing

2D and 3D surface image processing algorithms and their applications

This doctoral dissertation work aims to develop algorithms for 2D image segmentation application of solar filament disappearance detection, 3D mesh simplification, and 3D image warping in pre-surgery simulation. Filament area detection in solar images is an image segmentation problem. A thresholding and region growing combined method is proposed and applied in this application. Based on the filament area detection results, filament disappearances are reported in real time. The solar images in 1999 are processed with this proposed system and three statistical results of filaments are presented. 3D images can be obtained by passive and active range sensing. An image registration process finds the transformation between each pair of range views. To model an object, a common reference frame in which all views can be transformed must be defined. After the registration, the range views should be integrated into a non-redundant model. Optimization is necessary to obtain a complete 3D model. One single surface representation can better fit to the data. It may be further simplified for rendering, storing and transmitting efficiently, or the representation can be converted to some other formats. This work proposes an efficient algorithm for solving the mesh simplification problem, approximating an arbitrary mesh by a simplified mesh. The algorithm uses Root Mean Square distance error metric to decide the facet curvature. Two vertices of one edge and the surrounding vertices decide the average plane. The simplification results are excellent and the computation speed is fast. The algorithm is compared with six other major simplification algorithms. Image morphing is used for all methods that gradually and continuously deform a source image into a target image, while producing the in-between models. Image warping is a continuous deformation of a: graphical object. A morphing process is usually composed of warping and interpolation. This work develops a direct-manipulation-of-free-form-deformation-based method and application for pre-surgical planning. The developed user interface provides a friendly interactive tool in the plastic surgery. Nose augmentation surgery is presented as an example. Displacement vector and lattices resulting in different resolution are used to obtain various deformation results. During the deformation, the volume change of the model is also considered based on a simplified skin-muscle model

Perceptually Valid Facial Expressions for Character-Based Applications

This paper addresses the problem of creating facial expression of mixed emotions in a perceptually valid way. The research has been done in the context of a “game-like” health and education applications aimed at studying social competency and facial expression awareness in autistic children as well as native language learning, but the results can be applied to many other applications such as games with need for dynamic facial expressions or tools for automating the creation of facial animations. Most existing methods for creating facial expressions of mixed emotions use operations like averaging to create the combined effect of two universal emotions. Such methods may be mathematically justifiable but are not necessarily valid from a perceptual point of view. The research reported here starts by user experiments aiming at understanding how people combine facial actions to express mixed emotions, and how the viewers perceive a set of facial actions in terms of underlying emotions. Using the results of these experiments and a three-dimensional emotion model, we associate facial actions to dimensions and regions in the emotion space, and create a facial expression based on the location of the mixed emotion in the three-dimensional space. We call these regionalized facial actions “facial expression units.

Animating facial images with drawings

Ankara : Bilkent Univ., 1996.Thesis (Master's) -- Bilkent University, 1996.Includes bibliographical references leaves 54-56.The work presented here describes the power of 2D animation with texture mai^ping controlled by line drawings. Animation is specifically intended for facial animation and not restricted by the human face. We initially have a sequence of facial images which are taken from a video sequence of the same face and an image of another face to be animated. The aim is to animate the face image with the same expressions as those of the given video sequence. To realize the animation, a set of frames are taken from a video sequence. Key features of the first frame are I’otoscoped and the other frames are automatically rotoscoped using the first frame. Similarly, the corresponding features of the image which will be animated are rotoscoped. The key features of the first frame of the sequence and the image to be animated are mapped and using cross-synthesis procedure, other drawings for the given image are produced. Using these animated line drawings and the original image, the corresponding frame sequence is produced by image warping. The resulting sequence has the same expressions as those of the video sequence. This work encourages the reuse of animated motion by gathering facial motion sequences into a database. Furthermore, by using motion sequences of a human face, non-human characters can be animated realistically or complex characters can be animated by the help of motion sequences of simpler characters.Tunali, Gamze DilekM.S

Auto Lip-Sync Pada Karakter Virtual 3 Dimensi Menggunakan Blendshape

Proses pembuatan karakter virtual 3D yang dapat berbicara seperti manusia merupakan tantangan tersendiri bagi animator. Problematika yang muncul adalah dibutuhkan waktu lama dalam proses pengerjaan serta kompleksitas dari berbagai macam fonem penyusun kalimat. Teknik auto lip-sync digunakan untuk melakukan pembentukan karakter virtual 3D yang dapat berbicara seperti manusia pada umumnya. Preston blair phoneme series dijadikan acuan sebagai pembentukan viseme dalam karakter. Proses pemecahan fonem dan sinkronisasi audio dalam software 3D menjadi tahapan akhir dalam proses pembentukan auto lip-sync dalam karakter virtual 3D. Auto Lip-Sync on 3D Virtual Character Using Blendshape. Process of making a 3D virtual character who can speak like humans is a challenge for the animators. The problem that arise is that it takes a long time in the process as well as the complexity of the various phonemes making up sentences. Auto lip-sync technique is used to make the formation of a 3D virtual character who can speak like humans in general. Preston Blair phoneme series used as the reference in forming viseme in character. The phonemes solving process and audio synchronization in 3D software becomes the final stage in the process of auto lip-sync in a 3D virtual character

Differential equation-based shape interpolation for surface blending and facial blendshapes.

Differential equation-based shape interpolation has been widely applied in geometric modelling and computer animation. It has the advantages of physics-based, good realism, easy obtaining of high- order continuity, strong ability in describing complicated shapes, and small data of geometric models. Among various applications of differential equation-based shape interpolation, surface blending and facial blendshapes are two active and important topics. Differential equation-based surface blending can be time-independent and time-dependent. Existing differential equation-based surface blending only tackles time-dependen

Enhanced facial expression using oxygenation absorption of facial skin

Facial skin appearance is affected by physical and physiological state of the skin. The facial expression especially the skin appearances are in constant mutability and dynamically changed as human behave, talk and stress. The color of skin is considered to be one of the key indicators for these symptoms. The skin color resolution is highly determined by the scattering and absorption of light within the skin layers. The concentration of chromophores in melanin and hemoglobin oxygenation in the blood plays a pivotal role. An improvement work on prior model to create a realistic textured three-dimensional (3D) facial model for animation is proposed. This thesis considers both surface and subsurface scattering capable of simulating the interaction of light with the human skin. Furthermore, six parameters are used in this research which are the amount of oxygenation, de-oxygenation, hemoglobin, melanin, oil and blend factor for different types of melanin in the skin to generate a perfect match to specific skin types. The proposed model is associated with Blend Shape Interpolation and Facial Action Coding System to create five basic facial emotional expressions namely anger, happy, neutral, sad and fear. Meanwhile, the correlation between blood oxygenation in changing facial skin color for basic natural emotional expressions are measured using the Pulse Oximetry and 3D skin analyzer. The data from different subjects with male and female under different number of partially extreme facial expressions are fed in the model for simulation. The multi-pole method for layered materials is used to calculate the spectral diffusion profiles of two-layered skin which are further utilized to simulate the subsurface scattering of light within the skin. While the subsurface scattering is further combined with the Torrance-Sparrow Bidirectional Reflectance Distribution Function (BRDF) model to simulate the interaction of light with an oily layer at the skin surface. The result is validated by an evaluation procedure for measuring the accountability of a facial model via expressions and skin color of proposed model to the real human. The facial expressions evaluation is verified by calculating Euclidean distance between the facial markers of the real human and the avatar. The second assessment validates the skin color of facial expressions for the proposed avatar via the extraction of Histogram Color Features and Color Coherence Vector of each image with the real human and the previous work. The experimental result shows around 5.12 percent improvement compared to previous work. In achieving the realistic facial expression for virtual human based on facial skin color, texture and oxygenation of hemoglobin, the result demonstrates that the proposed model is beneficial to the development of virtual reality and game environment of computer aided graphics animation systems

Expressive Modulation of Neutral Visual Speech

The need for animated graphical models of the human face is commonplace in the movies, video games and television industries, appearing in everything from low budget advertisements and free mobile apps, to Hollywood blockbusters costing hundreds of millions of dollars. Generative statistical models of animation attempt to address some of the drawbacks of industry standard practices such as labour intensity and creative inflexibility. This work describes one such method for transforming speech animation curves between different expressive styles. Beginning with the assumption that expressive speech animation is a mix of two components, a high-frequency speech component (the content) and a much lower-frequency expressive component (the style), we use Independent Component Analysis (ICA) to identify and manipulate these components independently of one another. Next we learn how the energy for different speaking styles is distributed in terms of the low-dimensional independent components model. Transforming the speaking style involves projecting new animation curves into the lowdimensional ICA space, redistributing the energy in the independent components, and finally reconstructing the animation curves by inverting the projection. We show that a single ICA model can be used for separating multiple expressive styles into their component parts. Subjective evaluations show that viewers can reliably identify the expressive style generated using our approach, and that they have difficulty in identifying transformed animated expressive speech from the equivalent ground-truth

Trainable videorealistic speech animation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2002.Includes bibliographical references (p. 53-58).I describe how to create with machine learning techniques a generative, videorealistic, speech animation module. A human subject is first recorded using a videocamera as he/she utters a pre-determined speech corpus. After processing the corpus automatically, a visual speech module is learned from the data that is capable of synthesizing the human subject's mouth uttering entirely novel utterances that were not recorded in the original video. The synthesized utterance is re-composited onto a background sequence which contains natural head and eye movement. The final output is videorealistic in the sense that it looks like a video camera recording of the subject. At run time, the input to the system can be either real audio sequences or synthetic audio produced by a text-to-speech system, as long as they have been phonetically aligned. The two key contributions of this work are * a variant of the multidimensional morphable model (MMM) [4] [26] [25] to synthesize new, previously unseen mouth configurations from a small set of mouth image prototypes, * a trajectory synthesis technique based on regularization, which is automatically trained from the recorded video corpus, and which is capable of synthesizing trajectories in MMM space corresponding to any desired utterance. Results are presented on a series of numerical and psychophysical experiments designed to evaluate the synthetic animations.by Tony Farid Ezzat.Ph.D