280 research outputs found
Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors
We propose a new method for learning a generalized animatable neural human
representation from a sparse set of multi-view imagery of multiple persons. The
learned representation can be used to synthesize novel view images of an
arbitrary person from a sparse set of cameras, and further animate them with
the user's pose control. While existing methods can either generalize to new
persons or synthesize animations with user control, none of them can achieve
both at the same time. We attribute this accomplishment to the employment of a
3D proxy for a shared multi-person human model, and further the warping of the
spaces of different poses to a shared canonical pose space, in which we learn a
neural field and predict the person- and pose-dependent deformations, as well
as appearance with the features extracted from input images. To cope with the
complexity of the large variations in body shapes, poses, and clothing
deformations, we design our neural human model with disentangled geometry and
appearance. Furthermore, we utilize the image features both at the spatial
point and on the surface points of the 3D proxy for predicting person- and
pose-dependent properties. Experiments show that our method significantly
outperforms the state-of-the-arts on both tasks. The video and code are
available at https://talegqz.github.io/neural_novel_actor
HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video
We introduce HOSNeRF, a novel 360{\deg} free-viewpoint rendering method that
reconstructs neural radiance fields for dynamic human-object-scene from a
single monocular in-the-wild video. Our method enables pausing the video at any
frame and rendering all scene details (dynamic humans, objects, and
backgrounds) from arbitrary viewpoints. The first challenge in this task is the
complex object motions in human-object interactions, which we tackle by
introducing the new object bones into the conventional human skeleton hierarchy
to effectively estimate large object deformations in our dynamic human-object
model. The second challenge is that humans interact with different objects at
different times, for which we introduce two new learnable object state
embeddings that can be used as conditions for learning our human-object
representation and scene representation, respectively. Extensive experiments
show that HOSNeRF significantly outperforms SOTA approaches on two challenging
datasets by a large margin of 40% ~ 50% in terms of LPIPS. The code, data, and
compelling examples of 360{\deg} free-viewpoint renderings from single videos
will be released in https://showlab.github.io/HOSNeRF.Comment: Project page: https://showlab.github.io/HOSNeR
Mixed Reality Images: Trilogy of Synthetic Realities III
The interplay of physical reality and digital media technologies is getting enhanced by new interfaces. The age of hyper-reality turns into the age of hyper-aesthetics and immersive image technologies - like mixed reality - enable a completely novel form of interaction and user relation with the virtual image structures, the different screen technologies, and embedded physical artefacts for interaction. "Mixed Reality Images" contributes to the wide range of the hyper-aesthetic image discourse to connect the concept of mixed reality images with the approaches in modern media theory, philosophy, perceptual theory, aesthetics, computer graphics and art theory as well as the complex range of image science. This volume monitors and discusses the relation of images and technological evolution in the context of mixed reality within the perspective of an autonomous image science
Fictional Practices of Spirituality I: Interactive Media
"Fictional Practices of Spirituality" provides critical insight into the implementation of belief, mysticism, religion, and spirituality into worlds of fiction, be it interactive or non-interactive. This first volume focuses on interactive, virtual worlds - may that be the digital realms of video games and VR applications or the imaginary spaces of life action role-playing and soul-searching practices. It features analyses of spirituality as gameplay facilitator, sacred spaces and architecture in video game geography, religion in video games and spiritual acts and their dramaturgic function in video games, tabletop, or LARP, among other topics. The contributors offer a first-time ever comprehensive overview of play-rites as spiritual incentives and playful spirituality in various medial incarnations
Multilingual video dubbing—a technology review and current challenges
The proliferation of multi-lingual content on today’s streaming services has created a need for automated multi-lingual dubbing tools. In this article, current state-of-the-art approaches are discussed with reference to recent works in automatic dubbing and the closely related field of talking head generation. A taxonomy of papers within both fields is presented, and the main challenges of both speech-driven automatic dubbing, and talking head generation are discussed and outlined, together with proposals for future research to tackle these issues
2014 GREAT Day Program
SUNY Geneseo’s Eighth Annual GREAT Day.https://knightscholar.geneseo.edu/program-2007/1008/thumbnail.jp
High-fidelity Interpretable Inverse Rig: An Accurate and Sparse Solution Optimizing the Quartic Blendshape Model
We propose a method to fit arbitrarily accurate blendshape rig models by
solving the inverse rig problem in realistic human face animation. The method
considers blendshape models with different levels of added corrections and
solves the regularized least-squares problem using coordinate descent, i.e.,
iteratively estimating blendshape weights. Besides making the optimization
easier to solve, this approach ensures that mutually exclusive controllers will
not be activated simultaneously and improves the goodness of fit after each
iteration. We show experimentally that the proposed method yields solutions
with mesh error comparable to or lower than the state-of-the-art approaches
while significantly reducing the cardinality of the weight vector (over 20
percent), hence giving a high-fidelity reconstruction of the reference
expression that is easier to manipulate in the post-production manually. Python
scripts for the algorithm will be publicly available upon acceptance of the
paper
Deep Insights of Deepfake Technology : A Review
Under the aegis of computer vision and deep learning technology, a new
emerging techniques has introduced that anyone can make highly realistic but
fake videos, images even can manipulates the voices. This technology is widely
known as Deepfake Technology. Although it seems interesting techniques to make
fake videos or image of something or some individuals but it could spread as
misinformation via internet. Deepfake contents could be dangerous for
individuals as well as for our communities, organizations, countries religions
etc. As Deepfake content creation involve a high level expertise with
combination of several algorithms of deep learning, it seems almost real and
genuine and difficult to differentiate. In this paper, a wide range of articles
have been examined to understand Deepfake technology more extensively. We have
examined several articles to find some insights such as what is Deepfake, who
are responsible for this, is there any benefits of Deepfake and what are the
challenges of this technology. We have also examined several creation and
detection techniques. Our study revealed that although Deepfake is a threat to
our societies, proper measures and strict regulations could prevent this
Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
Deep person generation has attracted extensive research attention due to its
wide applications in virtual agents, video conferencing, online shopping and
art/movie production. With the advancement of deep learning, visual appearances
(face, pose, cloth) of a person image can be easily generated or manipulated on
demand. In this survey, we first summarize the scope of person generation, and
then systematically review recent progress and technical trends in deep person
generation, covering three major tasks: talking-head generation (face),
pose-guided person generation (pose) and garment-oriented person generation
(cloth). More than two hundred papers are covered for a thorough overview, and
the milestone works are highlighted to witness the major technical
breakthrough. Based on these fundamental tasks, a number of applications are
investigated, e.g., virtual fitting, digital human, generative data
augmentation. We hope this survey could shed some light on the future prospects
of deep person generation, and provide a helpful foundation for full
applications towards digital human
- …