Applications of Face Analysis and Modeling in Media Production

Facial expressions play an important role in day-by-day communication as well as media production. This article surveys automatic facial analysis and modeling methods using computer vision techniques and their applications for media production. The authors give a brief overview of the psychology of face perception and then describe some of the applications of computer vision and pattern recognition applied to face recognition in media production. This article also covers the automatic generation of face models, which are used in movie and TV productions for special effects in order to manipulate people's faces or combine real actors with computer graphics

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

We introduce FaceVR, a novel method for gaze-aware facial reenactment in the Virtual Reality (VR) context. The key component of FaceVR is a robust algorithm to perform real-time facial motion capture of an actor who is wearing a head-mounted display (HMD), as well as a new data-driven approach for eye tracking from monocular videos. In addition to these face reconstruction components, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances. For instance, we can alter facial expressions, change gaze directions, or remove the VR goggles in realistic re-renderings. In a live setup with a source and a target actor, we apply these newly-introduced algorithmic components. We assume that the source actor is wearing a VR device, and we capture his facial expressions and eye movement in real-time. For the target video, we mimic a similar tracking process; however, we use the source input to drive the animations of the target video, thus enabling gaze-aware facial reenactment. To render the modified target video on a stereo display, we augment our capture and reconstruction process with stereo data. In the end, FaceVR produces compelling results for a variety of applications, such as gaze-aware facial reenactment, reenactment in virtual reality, removal of VR goggles, and re-targeting of somebody's gaze direction in a video conferencing call

Applications of face analysis and modelling in media production:Overview of the state of the art

Facial expressions play an important role in day-by-day communication as well as media production. This article surveys automatic facial analysis and modeling methods using computer vision techniques and their applications for media production. The authors give a brief overview of the psychology of face perception and then describe some of the applications of computer vision and pattern recognition applied to face recognition in media production. This article also covers the automatic generation of face models, which are used in movie and TV productions for special effects in order to manipulate people's faces or combine real actors with computer graphics

A FACS Valid 3D Dynamic Action Unit Database with Applications to 3D Dynamic Morphable Facial Modeling

This paper presents the first dynamic 3D FACS data set for facial expression research, containing 10 subjects performing between 19 and 97 different AUs both individually and in combination. In total the corpus contains 519 AU sequences. The peak expression frame of each sequence has been manually FACS coded by certified FACS experts. This provides a ground truth for 3D FACS based AU recognition systems. In order to use this data, we describe the first framework for building dynamic 3D morphable models. This includes a novel Active Appearance Model (AAM) based 3D facial registration and mesh correspondence scheme. The approach overcomes limitations in existing methods that require facial markers or are prone to optical flow drift. We provide the first quantitative assessment of such 3D facial mesh registration techniques and show how our proposed method provides more reliable correspondence

FROM CELLULOID REALITIES TO BINARY DREAMSCAPES: CINEMA AND PERCEPTUAL EXPERIENCE IN THE AGE OF DIGITAL IMMERSION

Technologies in digital cinema are quickly changing the way contemporary filmmakers create films and how audiences currently perceive them. As we move onward into the digital turn, it becomes ever more apparent that the medium of film has been emancipated from its dependence on the photograph. Directors are no longer required to capture the objectively real as it sits before the photographic lens, but can essentially construct it via groundbreaking advancements in computer-generated imagery, motion capture technology, and digital 3D camera systems and display technologies. Since the origins of film, spectators and filmmakers have assumed an existing relationship between reality and the photographic image. Yet digital film technologies now provide us with hyper-facsimiles of reality that are perceived as photographic, but are often created by way of computer processes. Digital cinema currently allows the viewer to inhabit and interact with cinematic realities in unprecedented ways, and it is this contemporary paradigmatic shift from the analog to the digital that has catalyzed fundamentally new ways of looking at the filmic image. In this paper, I will examine the perceptual complexities of contemporary digital film through the lens of these cinematic technologies by examining their impact on the viewer’s experience