FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion

Speech-driven 3D facial animation synthesis has been a challenging task both in industry and research. Recent methods mostly focus on deterministic deep learning methods meaning that given a speech input, the output is always the same. However, in reality, the non-verbal facial cues that reside throughout the face are non-deterministic in nature. In addition, majority of the approaches focus on 3D vertex based datasets and methods that are compatible with existing facial animation pipelines with rigged characters is scarce. To eliminate these issues, we present FaceDiffuser, a non-deterministic deep learning model to generate speech-driven facial animations that is trained with both 3D vertex and blendshape based datasets. Our method is based on the diffusion technique and uses the pre-trained large speech representation model HuBERT to encode the audio input. To the best of our knowledge, we are the first to employ the diffusion method for the task of speech-driven 3D facial animation synthesis. We have run extensive objective and subjective analyses and show that our approach achieves better or comparable results in comparison to the state-of-the-art methods. We also introduce a new in-house dataset that is based on a blendshape based rigged character. We recommend watching the accompanying supplementary video. The code and the dataset will be publicly available.Comment: Pre-print of the paper accepted at ACM SIGGRAPH MIG 202

Shape Animation with Combined Captured and Simulated Dynamics

We present a novel volumetric animation generation framework to create new types of animations from raw 3D surface or point cloud sequence of captured real performances. The framework considers as input time incoherent 3D observations of a moving shape, and is thus particularly suitable for the output of performance capture platforms. In our system, a suitable virtual representation of the actor is built from real captures that allows seamless combination and simulation with virtual external forces and objects, in which the original captured actor can be reshaped, disassembled or reassembled from user-specified virtual physics. Instead of using the dominant surface-based geometric representation of the capture, which is less suitable for volumetric effects, our pipeline exploits Centroidal Voronoi tessellation decompositions as unified volumetric representation of the real captured actor, which we show can be used seamlessly as a building block for all processing stages, from capture and tracking to virtual physic simulation. The representation makes no human specific assumption and can be used to capture and re-simulate the actor with props or other moving scenery elements. We demonstrate the potential of this pipeline for virtual reanimation of a real captured event with various unprecedented volumetric visual effects, such as volumetric distortion, erosion, morphing, gravity pull, or collisions

Video Data Visualization System: Semantic Classification And Personalization

We present in this paper an intelligent video data visualization tool, based on semantic classification, for retrieving and exploring a large scale corpus of videos. Our work is based on semantic classification resulting from semantic analysis of video. The obtained classes will be projected in the visualization space. The graph is represented by nodes and edges, the nodes are the keyframes of video documents and the edges are the relation between documents and the classes of documents. Finally, we construct the user's profile, based on the interaction with the system, to render the system more adequate to its references.Comment: graphic

Graphic study and geovisualization of the old windmills of La Mancha (Spain)

In Spain, one can find geographical diversity and unique sites of great significance and cultural heritage. Many of the nation’s treasured places, however, have deteriorated or have even disappeared. What is left, then, should be studied and documented both graphically and infographically. It is important to preserve and document Spain’s unique locations, especially those related to vernacular heritage, to transhumance and visual impact assessment in many national infrastructures projects. Windmills are important examples of agro-industrial heritage and are sometimes found in the beds of streams and rivers but can also be found high in the hills. Their presence is constant throughout the Iberian Peninsula. These mills are no longer in use due to technological advances and the emergence of new grinding systems. The aim of this study was to present a specific methodology for the documentation of windmills, to create a graphical representation using computer graphics, as well as to disseminate knowledge of this agro-industrial heritage. This research has integrated graphic materials, including freehand sketches, photographs, digital orthophotos, computer graphics and multimedia in the creation of a specific methodology based on cutting-edge technology such as a digital photogrammetric workstation (DPW), global navigation satellite systems (GNSS), computer-aided design (CAD) and computer animation

How has my learning enabled me to create and share an animated video to assist newly qualified teachers in the creation of a safe critical space for their students?

This paper presents my emergent living theory as it developed while I attempted to address my concern “How has my learning enabled me to create and share an animated video to assist Newly Qualified Teachers in the creation of a safe critical space for their students?” I explore how my learning on the Masters in Education and Training Management (e-learning) has affected me both personally and professionally and how my learning has impacted the learning of others. In particular, I look at the role of critical pedagogy as it impacted my development of the video. I document the creation and distribution of an animated video using the State package by Xtranormal, through one action reflection cycle. I claim that through the use of an animated educational video, NQTs may engage critically with classroom management and in so doing improve their practice so that they can facilitate critical discourse among their students in a safe environment

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented