6 research outputs found
FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances
Visual effects commonly requires both the creation of realistic synthetic
humans as well as retargeting actors' performances to humanoid characters such
as aliens and monsters. Achieving the expressive performances demanded in
entertainment requires manipulating complex models with hundreds of parameters.
Full creative control requires the freedom to make edits at any stage of the
production, which prohibits the use of a fully automatic ``black box'' solution
with uninterpretable parameters. On the other hand, producing realistic
animation with these sophisticated models is difficult and laborious. This
paper describes FDLS (Facial Deep Learning Solver), which is Weta Digital's
solution to these challenges. FDLS adopts a coarse-to-fine and
human-in-the-loop strategy, allowing a solved performance to be verified and
edited at several stages in the solving process. To train FDLS, we first
transform the raw motion-captured data into robust graph features. Secondly,
based on the observation that the artists typically finalize the jaw pass
animation before proceeding to finer detail, we solve for the jaw motion first
and predict fine expressions with region-based networks conditioned on the jaw
position. Finally, artists can optionally invoke a non-linear finetuning
process on top of the FDLS solution to follow the motion-captured virtual
markers as closely as possible. FDLS supports editing if needed to improve the
results of the deep learning solution and it can handle small daily changes in
the actor's face shape. FDLS permits reliable and production-quality
performance solving with minimal training and little or no manual effort in
many cases, while also allowing the solve to be guided and edited in unusual
and difficult cases. The system has been under development for several years
and has been used in major movies.Comment: DigiPro '22: The Digital Production Symposiu
Intermediate cells of in vitro cellular reprogramming and in vivo tissue regeneration require desmoplakin
Amphibians and fish show considerable regeneration potential via dedifferentiation of somatic cells into blastemal cells. In terms of dedifferentiation, in vitro cellular reprogramming has been proposed to share common processes with in vivo tissue regeneration, although the details are elusive. Here, we identified the cytoskeletal linker protein desmoplakin (Dsp) as a common factor mediating both reprogramming and regeneration. Our analysis revealed that Dsp expression is elevated in distinct intermediate cells during in vitro reprogramming. Knockdown of Dsp impedes in vitro reprogramming into induced pluripotent stem cells and induced neural stem/progenitor cells as well as in vivo regeneration of zebrafish fins. Notably, reduced Dsp expression impairs formation of the intermediate cells during cellular reprogramming and tissue regeneration. These findings suggest that there is a Dsp-mediated evolutionary link between cellular reprogramming in mammals and tissue regeneration in lower vertebrates and that the intermediate cells may provide alternative approaches for mammalian regenerative therapy.11Nsciescopu