10,685 research outputs found
Shape Animation with Combined Captured and Simulated Dynamics
We present a novel volumetric animation generation framework to create new
types of animations from raw 3D surface or point cloud sequence of captured
real performances. The framework considers as input time incoherent 3D
observations of a moving shape, and is thus particularly suitable for the
output of performance capture platforms. In our system, a suitable virtual
representation of the actor is built from real captures that allows seamless
combination and simulation with virtual external forces and objects, in which
the original captured actor can be reshaped, disassembled or reassembled from
user-specified virtual physics. Instead of using the dominant surface-based
geometric representation of the capture, which is less suitable for volumetric
effects, our pipeline exploits Centroidal Voronoi tessellation decompositions
as unified volumetric representation of the real captured actor, which we show
can be used seamlessly as a building block for all processing stages, from
capture and tracking to virtual physic simulation. The representation makes no
human specific assumption and can be used to capture and re-simulate the actor
with props or other moving scenery elements. We demonstrate the potential of
this pipeline for virtual reanimation of a real captured event with various
unprecedented volumetric visual effects, such as volumetric distortion,
erosion, morphing, gravity pull, or collisions
Deep Autoencoder for Combined Human Pose Estimation and body Model Upscaling
We present a method for simultaneously estimating 3D human pose and body
shape from a sparse set of wide-baseline camera views. We train a symmetric
convolutional autoencoder with a dual loss that enforces learning of a latent
representation that encodes skeletal joint positions, and at the same time
learns a deep representation of volumetric body shape. We harness the latter to
up-scale input volumetric data by a factor of , whilst recovering a
3D estimate of joint positions with equal or greater accuracy than the state of
the art. Inference runs in real-time (25 fps) and has the potential for passive
human behaviour monitoring where there is a requirement for high fidelity
estimation of human body shape and pose
RGBD Datasets: Past, Present and Future
Since the launch of the Microsoft Kinect, scores of RGBD datasets have been
released. These have propelled advances in areas from reconstruction to gesture
recognition. In this paper we explore the field, reviewing datasets across
eight categories: semantics, object pose estimation, camera tracking, scene
reconstruction, object tracking, human actions, faces and identification. By
extracting relevant information in each category we help researchers to find
appropriate data for their needs, and we consider which datasets have succeeded
in driving computer vision forward and why.
Finally, we examine the future of RGBD datasets. We identify key areas which
are currently underexplored, and suggest that future directions may include
synthetic data and dense reconstructions of static and dynamic scenes.Comment: 8 pages excluding references (CVPR style
Robust Temporally Coherent Laplacian Protrusion Segmentation of 3D Articulated Bodies
In motion analysis and understanding it is important to be able to fit a
suitable model or structure to the temporal series of observed data, in order
to describe motion patterns in a compact way, and to discriminate between them.
In an unsupervised context, i.e., no prior model of the moving object(s) is
available, such a structure has to be learned from the data in a bottom-up
fashion. In recent times, volumetric approaches in which the motion is captured
from a number of cameras and a voxel-set representation of the body is built
from the camera views, have gained ground due to attractive features such as
inherent view-invariance and robustness to occlusions. Automatic, unsupervised
segmentation of moving bodies along entire sequences, in a temporally-coherent
and robust way, has the potential to provide a means of constructing a
bottom-up model of the moving body, and track motion cues that may be later
exploited for motion classification. Spectral methods such as locally linear
embedding (LLE) can be useful in this context, as they preserve "protrusions",
i.e., high-curvature regions of the 3D volume, of articulated shapes, while
improving their separation in a lower dimensional space, making them in this
way easier to cluster. In this paper we therefore propose a spectral approach
to unsupervised and temporally-coherent body-protrusion segmentation along time
sequences. Volumetric shapes are clustered in an embedding space, clusters are
propagated in time to ensure coherence, and merged or split to accommodate
changes in the body's topology. Experiments on both synthetic and real
sequences of dense voxel-set data are shown. This supports the ability of the
proposed method to cluster body-parts consistently over time in a totally
unsupervised fashion, its robustness to sampling density and shape quality, and
its potential for bottom-up model constructionComment: 31 pages, 26 figure
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at
Siggraph'1
Evaluating the accuracy of diffusion MRI models in white matter
Models of diffusion MRI within a voxel are useful for making inferences about
the properties of the tissue and inferring fiber orientation distribution used
by tractography algorithms. A useful model must fit the data accurately.
However, evaluations of model-accuracy of some of the models that are commonly
used in analyzing human white matter have not been published before. Here, we
evaluate model-accuracy of the two main classes of diffusion MRI models. The
diffusion tensor model (DTM) summarizes diffusion as a 3-dimensional Gaussian
distribution. Sparse fascicle models (SFM) summarize the signal as a linear sum
of signals originating from a collection of fascicles oriented in different
directions. We use cross-validation to assess model-accuracy at different
gradient amplitudes (b-values) throughout the white matter. Specifically, we
fit each model to all the white matter voxels in one data set and then use the
model to predict a second, independent data set. This is the first evaluation
of model-accuracy of these models. In most of the white matter the DTM predicts
the data more accurately than test-retest reliability; SFM model-accuracy is
higher than test-retest reliability and also higher than the DTM, particularly
for measurements with (a) a b-value above 1000 in locations containing fiber
crossings, and (b) in the regions of the brain surrounding the optic
radiations. The SFM also has better parameter-validity: it more accurately
estimates the fiber orientation distribution function (fODF) in each voxel,
which is useful for fiber tracking
Experimental Design Modulates Variance in BOLD Activation: The Variance Design General Linear Model
Typical fMRI studies have focused on either the mean trend in the
blood-oxygen-level-dependent (BOLD) time course or functional connectivity
(FC). However, other statistics of the neuroimaging data may contain important
information. Despite studies showing links between the variance in the BOLD
time series (BV) and age and cognitive performance, a formal framework for
testing these effects has not yet been developed. We introduce the Variance
Design General Linear Model (VDGLM), a novel framework that facilitates the
detection of variance effects. We designed the framework for general use in any
fMRI study by modeling both mean and variance in BOLD activation as a function
of experimental design. The flexibility of this approach allows the VDGLM to i)
simultaneously make inferences about a mean or variance effect while
controlling for the other and ii) test for variance effects that could be
associated with multiple conditions and/or noise regressors. We demonstrate the
use of the VDGLM in a working memory application and show that engagement in a
working memory task is associated with whole-brain decreases in BOLD variance.Comment: 18 pages, 7 figure
- …