140 research outputs found

    A sketch-based interface for facial animation in immersive virtual reality

    No full text
    Creating facial animations using 3D computer graphics represents a very laborious and time-consuming task. Among the numberless approaches for animating faces, the use of blendshapes remains the most common solution because of their simplic- ity and the ability to produce high-quality results. This approach, however, is also characterized by important drawbacks. With the traditional animation suites, in order to select the blendshapes to be activated animators are generally requested to memorize the mapping between the blendshapes and the influenced mesh vertices; alternatively, they need to adopt a trial-and-error search within the whole library of available blendshapes. Moreover, the level of expressiveness that can be reached may be lower than expected; this is due to the fact that the possibility to apply transformations to vertices different than just linear translations and mechanisms for adding, e.g., exaggerations, are typically not integrated into the same anima- tion environment. In order to tackle these issues, this paper proposes an immersive virtual reality-based interface that leverages sketches for the direct manipulation of blendshapes. Animators can draw both linear and curved strokes, which are used to automatically extract information about the blendshape to be activated and its weight, the trajectories that associated vertices have to follow, as well as the timing of the overall animation. A user study was carried out with the aim of evaluating the proposed approach onto several representative animations tasks. Both objective and subjective measurements were collected. Experimental results showed the benefits of the devised interface in terms of task completion time, animation accuracy, and usability

    Exploring Virtual Reality and Doppelganger Avatars for the Treatment of Chronic Back Pain

    Get PDF
    Cognitive-behavioral models of chronic pain assume that fear of pain and subsequent avoidance behavior contribute to pain chronicity and the maintenance of chronic pain. In chronic back pain (CBP), avoidance of movements often plays a major role in pain perseverance and interference with daily life activities. In treatment, avoidance is often addressed by teaching patients to reduce pain behaviors and increase healthy behaviors. The current project explored the use of personalized virtual characters (doppelganger avatars) in virtual reality (VR), to influence motor imitation and avoidance, fear of pain and experienced pain in CBP. We developed a method to create virtual doppelgangers, to animate them with movements captured from real-world models, and to present them to participants in an immersive cave virtual environment (CAVE) as autonomous movement models for imitation. Study 1 investigated interactions between model and observer characteristics in imitation behavior of healthy participants. We tested the hypothesis that perceived affiliative characteristics of a virtual model, such as similarity to the observer and likeability, would facilitate observers’ engagement in voluntary motor imitation. In a within-subject design (N=33), participants were exposed to four virtual characters of different degrees of realism and observer similarity, ranging from an abstract stickperson to a personalized doppelganger avatar designed from 3d scans of the observer. The characters performed different trunk movements and participants were asked to imitate these. We defined functional ranges of motion (ROM) for spinal extension (bending backward, BB), lateral flexion (bending sideward, BS) and rotation in the horizontal plane (RH) based on shoulder marker trajectories as behavioral indicators of imitation. Participants’ ratings on perceived avatar appearance were recorded in an Autonomous Avatar Questionnaire (AAQ), based on an explorative factor analysis. Linear mixed effects models revealed that for lateral flexion (BS), a facilitating influence of avatar type on ROM was mediated by perceived identification with the avatar including avatar likeability, avatar-observer-similarity and other affiliative characteristics. These findings suggest that maximizing model-observer similarity may indeed be useful to stimulate observational modeling. Study 2 employed the techniques developed in study 1 with participants who suffered from CBP and extended the setup with real-world elements, creating an immersive mixed reality. The research question was whether virtual doppelgangers could modify motor behaviors, pain expectancy and pain. In a randomized controlled between-subject design, participants observed and imitated an avatar (AVA, N=17) or a videotaped model (VID, N=16) over three sessions, during which the movements BS and RH as well as a new movement (moving a beverage crate) were shown. Again, self-reports and ROMs were used as measures. The AVA group reported reduced avoidance with no significant group differences in ROM. Pain expectancy increased in AVA but not VID over the sessions. Pain and limitations did not significantly differ. We observed a moderation effect of group, with prior pain expectancy predicting pain and avoidance in the VID but not in the AVA group. This can be interpreted as an effect of personalized movement models decoupling pain behavior from movement-related fear and pain expectancy by increasing pain tolerance and task persistence. Our findings suggest that personalized virtual movement models can stimulate observational modeling in general, and that they can increase pain tolerance and persistence in chronic pain conditions. Thus, they may provide a tool for exposure and exercise treatments in cognitive behavioral treatment approaches to CBP

    High-fidelity Interpretable Inverse Rig: An Accurate and Sparse Solution Optimizing the Quartic Blendshape Model

    Full text link
    We propose a method to fit arbitrarily accurate blendshape rig models by solving the inverse rig problem in realistic human face animation. The method considers blendshape models with different levels of added corrections and solves the regularized least-squares problem using coordinate descent, i.e., iteratively estimating blendshape weights. Besides making the optimization easier to solve, this approach ensures that mutually exclusive controllers will not be activated simultaneously and improves the goodness of fit after each iteration. We show experimentally that the proposed method yields solutions with mesh error comparable to or lower than the state-of-the-art approaches while significantly reducing the cardinality of the weight vector (over 20 percent), hence giving a high-fidelity reconstruction of the reference expression that is easier to manipulate in the post-production manually. Python scripts for the algorithm will be publicly available upon acceptance of the paper

    Kaijus as environments: design & production of a colossal monster functioning as a boss level

    Get PDF
    Boss fights are a staple in most video game genres. They are milestones in the adventure, designed and intended to test the skills that the player has acquired throughout their adventure. In some cases, they even define the whole experience of the game, especially one type of enemy that has appeared in several instances and every genre: colossal bosses, monsters of giant proportions usually used as a matter of spectacle and a simple yet effective way to showcase the sheer power that players have achieved up until that point in the adventure. Titles like God of War, Shadow of the Colossus and even many Super Mario titles use this concept in their video games in imaginative ways to create Kaiju-like creatures working as a living environment the player has to traverse to defeat them. However, what is the process behind creating a colossal boss that works as a breathing environment, and how can it be achieved? This project aims to study the process of colossal boss creation and design and apply level design and asset creation. To do this, the author will investigate the main aspects and key-defining features of these bosses, analyzing the strengths and weaknesses of existing bosses in videogames such as God of War 3’s Cronos and Shadow of the Colossus and Solar Ash’s bosses in terms of art production and game design. From this study and following the art process for creating creatures in the video game industry, the author will conceptualize, design and produce a working, playable prototype of a boss fight, showcased in the final presentation

    Genetic algorithms reveal identity independent representation of emotional expressions

    Get PDF
    People readily and automatically process facial emotion and identity, and it has been reported that these cues are processed both dependently and independently. However, this question of identity independent encoding of emotions has only been examined using posed, often exaggerated expressions of emotion, that do not account for the substantial individual differences in emotion recognition. In this study, we ask whether people's unique beliefs of how emotions should be reflected in facial expressions depend on the identity of the face. To do this, we employed a genetic algorithm where participants created facial expressions to represent different emotions. Participants generated facial expressions of anger, fear, happiness, and sadness, on two different identities. Facial features were controlled by manipulating a set of weights, allowing us to probe the exact positions of faces in high-dimensional expression space. We found that participants created facial expressions belonging to each identity in a similar space that was unique to the participant, for angry, fearful, and happy expressions, but not sad. However, using a machine learning algorithm that examined the positions of faces in expression space, we also found systematic differences between the two identities' expressions across participants. This suggests that participants' beliefs of how an emotion should be reflected in a facial expression are unique to them and identity independent, although there are also some systematic differences in the facial expressions between two identities that are common across all individuals. (PsycInfo Database Record (c) 2023 APA, all rights reserved)

    Photo-realistic face synthesis and reenactment with deep generative models

    Get PDF
    The advent of Deep Learning has led to numerous breakthroughs in the field of Computer Vision. Over the last decade, a significant amount of research has been undertaken towards designing neural networks for visual data analysis. At the same time, rapid advancements have been made towards the direction of deep generative modeling, especially after the introduction of Generative Adversarial Networks (GANs), which have shown particularly promising results when it comes to synthesising visual data. Since then, considerable attention has been devoted to the problem of photo-realistic human face animation due to its wide range of applications, including image and video editing, virtual assistance, social media, teleconferencing, and augmented reality. The objective of this thesis is to make progress towards generating photo-realistic videos of human faces. To that end, we propose novel generative algorithms that provide explicit control over the facial expression and head pose of synthesised subjects. Despite the major advances in face reenactment and motion transfer, current methods struggle to generate video portraits that are indistinguishable from real data. In this work, we aim to overcome the limitations of existing approaches, by combining concepts from deep generative networks and video-to-video translation with 3D face modelling, and more specifically by capitalising on prior knowledge of faces that is enclosed within statistical models such as 3D Morphable Models (3DMMs). In the first part of this thesis, we introduce a person-specific system that performs full head reenactment using ideas from video-to-video translation. Subsequently, we propose a novel approach to controllable video portrait synthesis, inspired from Implicit Neural Representations (INR). In the second part of the thesis, we focus on person-agnostic methods and present a GAN-based framework that performs video portrait reconstruction, full head reenactment, expression editing, novel pose synthesis and face frontalisation.Open Acces

    Genetic algorithms reveal identity independent representation of emotional expressions.

    Get PDF
    People readily and automatically process facial emotion and identity, and it has been reported that these cues are processed both dependently and independently. However, this question of identity independent encoding of emotions has only been examined using posed, often exaggerated expressions of emotion, that do not account for the substantial individual differences in emotion recognition. In this study, we ask whether people's unique beliefs of how emotions should be reflected in facial expressions depend on the identity of the face. To do this, we employed a genetic algorithm where participants created facial expressions to represent different emotions. Participants generated facial expressions of anger, fear, happiness, and sadness, on two different identities. Facial features were controlled by manipulating a set of weights, allowing us to probe the exact positions of faces in high-dimensional expression space. We found that participants created facial expressions belonging to each identity in a similar space that was unique to the participant, for angry, fearful, and happy expressions, but not sad. However, using a machine learning algorithm that examined the positions of faces in expression space, we also found systematic differences between the two identities' expressions across participants. This suggests that participants' beliefs of how an emotion should be reflected in a facial expression are unique to them and identity independent, although there are also some systematic differences in the facial expressions between two identities that are common across all individuals. (PsycInfo Database Record (c) 2023 APA, all rights reserved)

    Distributed Solution of the Inverse Rig Problem in Blendshape Facial Animation

    Full text link
    The problem of rig inversion is central in facial animation as it allows for a realistic and appealing performance of avatars. With the increasing complexity of modern blendshape models, execution times increase beyond practically feasible solutions. A possible approach towards a faster solution is clustering, which exploits the spacial nature of the face, leading to a distributed method. In this paper, we go a step further, involving cluster coupling to get more confident estimates of the overlapping components. Our algorithm applies the Alternating Direction Method of Multipliers, sharing the overlapping weights between the subproblems. The results obtained with this technique show a clear advantage over the naive clustered approach, as measured in different metrics of success and visual inspection. The method applies to an arbitrary clustering of the face. We also introduce a novel method for choosing the number of clusters in a data-free manner. The method tends to find a clustering such that the resulting clustering graph is sparse but without losing essential information. Finally, we give a new variant of a data-free clustering algorithm that produces good scores with respect to the mentioned strategy for choosing the optimal clustering

    Face Reenactment with Generative Landmark Guidance

    Get PDF
    Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning
    corecore