ICface: Interpretable and Controllable Face Reenactment Using GANs

This paper presents a generic face animator that is able to control the pose and expressions of a given face image. The animation is driven by human interpretable control signals consisting of head pose angles and the Action Unit (AU) values. The control information can be obtained from multiple sources including external driving videos and manual controls. Due to the interpretable nature of the driving signal, one can easily mix the information between multiple sources (e.g. pose from one image and expression from another) and apply selective post-production editing. The proposed face animator is implemented as a two-stage neural network model that is learned in a self-supervised manner using a large video collection. The proposed Interpretable and Controllable face reenactment network (ICface) is compared to the state-of-the-art neural network-based face animation techniques in multiple tasks. The results indicate that ICface produces better visual quality while being more versatile than most of the comparison methods. The introduced model could provide a lightweight and easy to use tool for a multitude of advanced image and video editing tasks.Comment: Accepted in WACV-202

Guiding the Eye: A Non-photorealistic Solution for Controlling Viewer Interest

In film and still photography, depth of field control is often employed to control viewer interest in an image. This technique is also used in computer animation, but, in a medium where artists have near infinite control, must we rely on replicating photorealism? This research is a viable, non-photorealistic solution to the problem of directing viewer interest. Vision is directed by reducing superfluous visual information from parts of the image, which do not directly affect the depictive meaning of that image. This concept is applied to images and animations rendered from three-dimensional, computergenerated scenes, where detail is defined as visual information pertaining to the surface properties of a given object. A system is developed to demonstrate this concept. The system uses distance from a user-defined origin as the main mechanism to modulate detail. This solution is implemented within a modeling and shading environment to serve as a non-photorealistic, functional alternative for depth of field. This approach is conceptually based on a model of human vision, specifically, the relationship between foveal and peripheral vision, and is artistically driven by various works in the disciplines of painting and illustration, that through the careful manipulation of detail, control interest and understanding within the image. The resulting images and animations produced by this system provide viable evidence that detail modulation can be used to control effectively viewer interest in an image eliminating the need to use photographic techniques like depth of field

The Assistant Personal Robot Project: From the APR-01 to the APR-02 Mobile Robot Prototypes

This paper describes the evolution of the Assistant Personal Robot (APR) project developed at the Robotics Laboratory of the University of Lleida, Spain. This paper describes the first APR-01 prototype developed, the basic hardware improvement, the specific anthropomorphic improvements, and the preference surveys conducted with engineering students from the same university in order to maximize the perceived affinity with the final APR-02 mobile robot prototype. The anthropomorphic improvements have covered the design of the arms, the implementation of the arm and symbolic hand, the selection of a face for the mobile robot, the selection of a neutral facial expression, the selection of an animation for the mouth, the application of proximity feedback, the application of gaze feedback, the use of arm gestures, the selection of the motion planning strategy, and the selection of the nominal translational velocity. The final conclusion is that the development of preference surveys during the implementation of the APR-02 prototype has greatly influenced its evolution and has contributed to increase the perceived affinity and social acceptability of the prototype, which is now ready to develop assistance applications in dynamic workspaces.This research was partially funded by the Accessibility Chair promoted by Indra, Adecco Foundation and the University of Lleida Foundation from 2006 to 2018. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results

Virtual humans and Photorealism: The effect of photorealism of interactive virtual humans in clinical virtual environment on affective responses

The ability of realistic vs stylized representations of virtual characters to elicit emotions in users has been an open question for researchers and artists alike. We designed and performed a between subjects experiment using a medical virtual reality simulation to study the differences in the emotions aroused in participants while interacting with realistic and stylized virtual characters. The experiment included three conditions each of which presented a different representation of the virtual character namely; photo-realistic, non-photorealistic cartoon-shaded and non-photorealistic charcoal-sketch. The simulation used for the experiment, called the Rapid Response Training System was developed to train nurses to identify symptoms of rapid deterioration in patients. The emotional impact of interacting with the simulation on the participants was measured via both subjective and objective metrics. Quantitative objective measures were gathered using skin Electrodermal Activity (EDA) sensors, and quantitative subjective measures included Differential Emotion Survey (DES IV), Positive and Negative Affect Schedule (PANAS), and the co-presence or social presence questionnaire. The emotional state of the participants was analyzed across four distinct time steps during which the medical condition of the virtual patient deteriorated, and was contrasted to a baseline affective state. The data from the EDA sensors indicated that the mean level of arousal was highest in the charcoal-sketch condition, lowest in the realistic condition, with responses in the cartoon-shaded condition was in the middle. Mean arousal responses also seemed to be consistent in both the cartoon-shaded and charcoal-sketch conditions across all time steps, while the mean arousal response of participants in the realistic condition showed a significant drop from time step 1 through time step 2, corresponding to the deterioration of the virtual patient. Mean scores of participants in the DES survey seems to suggest that participants in the realistic condition elicited a higher emotional response than participants in both non-realistic conditions. Within the non-realistic conditions, participants in the cartoon-shaded condition seemed to elicit a higher emotional response than those in the charcoal-sketch condition

EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment

Face performance capture and reenactment techniques use multiple cameras and sensors, positioned at a distance from the face or mounted on heavy wearable devices. This limits their applications in mobile and outdoor environments. We present EgoFace, a radically new lightweight setup for face performance capture and front-view videorealistic reenactment using a single egocentric RGB camera. Our lightweight setup allows operations in uncontrolled environments, and lends itself to telepresence applications such as video-conferencing from dynamic environments. The input image is projected into a low dimensional latent space of the facial expression parameters. Through careful adversarial training of the parameter-space synthetic rendering, a videorealistic animation is produced. Our problem is challenging as the human visual system is sensitive to the smallest face irregularities that could occur in the final results. This sensitivity is even stronger for video results. Our solution is trained in a pre-processing stage, through a supervised manner without manual annotations. EgoFace captures a wide variety of facial expressions, including mouth movements and asymmetrical expressions. It works under varying illuminations, background, movements, handles people from different ethnicities and can operate in real time

Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-Identification

Through avatar embodiment in Virtual Reality (VR) we can achieve the illusion that an avatar is substituting our body: the avatar moves as we move and we see it from a first person perspective. However, self-identification, the process of identifying a representation as being oneself, poses new challenges because a key determinant is that we see and have agency in our own face. Providing control over the face is hard with current HMD technologies because face tracking is either cumbersome or error prone. However, limited animation is easily achieved based on speaking. We investigate the level of avatar enfacement, that is believing that a picture of a face is one's own face, with three levels of facial animation: (i) one in which the facial expressions of the avatars are static, (ii) one in which we implement lip-sync motion and (iii) one in which the avatar presents lip-sync plus additional facial animations, with blinks, designed by a professional animator. We measure self-identification using a face morphing tool that morphs from the face of the participant to the face of a gender matched avatar. We find that self-identification on avatars can be increased through pre-baked animations even when these are not photorealistic nor look like the participant

Gazedirector: Fully articulated eye gaze redirection in video

We present GazeDirector, a new approach for eye gaze redirection that uses model-fitting. Our method first tracks the eyes by fitting a multi-part eye region model to video frames using analysis-by-synthesis, thereby recovering eye region shape, texture, pose, and gaze simultaneously. It then redirects gaze by 1) warping the eyelids from the original image using a model-derived flow field, and 2) rendering and compositing synthesized 3D eyeballs onto the output image in a photorealistic manner. GazeDirector allows us to change where people are looking without person-specific training data, and with full articulation, i.e. we can precisely specify new gaze directions in 3D. Quantitatively, we evaluate both model-fitting and gaze synthesis, with experiments for gaze estimation and redirection on the Columbia gaze dataset. Qualitatively, we compare GazeDirector against recent work on gaze redirection, showing better results especially for large redirection angles. Finally, we demonstrate gaze redirection on YouTube videos by introducing new 3D gaze targets and by manipulating visual behavior