4,297 research outputs found
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at
Siggraph'1
DyNCA: Real-time Dynamic Texture Synthesis Using Neural Cellular Automata
Current Dynamic Texture Synthesis (DyTS) models in the literature can
synthesize realistic videos. However, these methods require a slow iterative
optimization process to synthesize a single fixed-size short video, and they do
not offer any post-training control over the synthesis process. We propose
Dynamic Neural Cellular Automata (DyNCA), a framework for real-time and
controllable dynamic texture synthesis. Our method is built upon the recently
introduced NCA models, and can synthesize infinitely-long and arbitrary-size
realistic texture videos in real-time. We quantitatively and qualitatively
evaluate our model and show that our synthesized videos appear more realistic
than the existing results. We improve the SOTA DyTS performance by
orders of magnitude. Moreover, our model offers several real-time and
interactive video controls including motion speed, motion direction, and an
editing brush tool
Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
Event cameras are bio-inspired vision sensors that output pixel-level
brightness changes instead of standard intensity frames. These cameras do not
suffer from motion blur and have a very high dynamic range, which enables them
to provide reliable visual information during high speed motions or in scenes
characterized by high dynamic range. However, event cameras output only little
information when the amount of motion is limited, such as in the case of almost
still motion. Conversely, standard cameras provide instant and rich information
about the environment most of the time (in low-speed and good lighting
scenarios), but they fail severely in case of fast motions, or difficult
lighting such as high dynamic range or low light scenes. In this paper, we
present the first state estimation pipeline that leverages the complementary
advantages of these two sensors by fusing in a tightly-coupled manner events,
standard frames, and inertial measurements. We show on the publicly available
Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement
of 130% over event-only pipelines, and 85% over standard-frames-only
visual-inertial systems, while still being computationally tractable.
Furthermore, we use our pipeline to demonstrate - to the best of our knowledge
- the first autonomous quadrotor flight using an event camera for state
estimation, unlocking flight scenarios that were not reachable with traditional
visual-inertial odometry, such as low-light environments and high-dynamic range
scenes.Comment: 8 pages, 9 figures, 2 table
Aerospace medicine and biology: A continuing bibliography with indexes, supplement 125
This special bibliography lists 323 reports, articles, and other documents introduced into the NASA scientific and technical information system in January 1974
SIGGRAPH
We present a method for recovering a temporally coherent, deforming triangle mesh with arbitrarily changing topology from an incoherent sequence of static closed surfaces. We solve this problem using the surface geometry alone, without any prior information like surface templates or velocity fields. Our system combines a proven strategy for triangle mesh improvement, a robust multi-resolution non-rigid registration routine, and a reliable technique for changing surface mesh topology. We also introduce a novel topological constraint enforcement algorithm to ensure that the output and input always have similar topology. We apply our technique to a series of diverse input data from video reconstructions, physics simulations, and artistic morphs. The structured output of our algorithm allows us to efficiently track information like colors and displacement maps, recover velocity information, and solve PDEs on the mesh as a post process
A Three-level Motion Texture for Human Motion Modeling
Abstract- A three-level motion texture is proposed to model complex human motion that is statistically similar to the original motion data. The three-level structure, namely moton index, moton and moton distribution, is defined to synthesize motions. To describe the continuous and non-linear dynamics of human motion, the motion texture is modeled by a Non-Stationary Switching Linear Dynamic System (NS-SLDS), which improves the Switching Linear Dynamic System (SLDS) by non-stationary functions. A BSK-tree (Binary Key-pose Splitting Tree) retrieval method applied in motons supplies the ability to access data in frame-level. Thus the motion texture can be manipulated at three different levels, by retrieving key-frame in specific moton, by changing the details of a specific motion at the moton-level and by designing a new choreography at the distribution-level. In motion synthesis experiments, the proposed approach was proved flexible and effective. Index Terms- motion texture. moton. NS-SLDS. KBS-tree. I
- …