98,827 research outputs found
Temporally coherent 4D reconstruction of complex dynamic scenes
This paper presents an approach for reconstruction of 4D temporally coherent
models of complex dynamic scenes. No prior knowledge is required of scene
structure or camera calibration allowing reconstruction from multiple moving
cameras. Sparse-to-dense temporal correspondence is integrated with joint
multi-view segmentation and reconstruction to obtain a complete 4D
representation of static and dynamic objects. Temporal coherence is exploited
to overcome visual ambiguities resulting in improved reconstruction of complex
scenes. Robust joint segmentation and reconstruction of dynamic objects is
achieved by introducing a geodesic star convexity constraint. Comparative
evaluation is performed on a variety of unstructured indoor and outdoor dynamic
scenes with hand-held cameras and multiple people. This demonstrates
reconstruction of complete temporally coherent 4D scene models with improved
nonrigid object segmentation and shape reconstruction.Comment: To appear in The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR) 2016 . Video available at:
https://www.youtube.com/watch?v=bm_P13_-Ds
Sampling and Recovery of Pulse Streams
Compressive Sensing (CS) is a new technique for the efficient acquisition of
signals, images, and other data that have a sparse representation in some
basis, frame, or dictionary. By sparse we mean that the N-dimensional basis
representation has just K<<N significant coefficients; in this case, the CS
theory maintains that just M = K log N random linear signal measurements will
both preserve all of the signal information and enable robust signal
reconstruction in polynomial time. In this paper, we extend the CS theory to
pulse stream data, which correspond to S-sparse signals/images that are
convolved with an unknown F-sparse pulse shape. Ignoring their convolutional
structure, a pulse stream signal is K=SF sparse. Such signals figure
prominently in a number of applications, from neuroscience to astronomy. Our
specific contributions are threefold. First, we propose a pulse stream signal
model and show that it is equivalent to an infinite union of subspaces. Second,
we derive a lower bound on the number of measurements M required to preserve
the essential information present in pulse streams. The bound is linear in the
total number of degrees of freedom S + F, which is significantly smaller than
the naive bound based on the total signal sparsity K=SF. Third, we develop an
efficient signal recovery algorithm that infers both the shape of the impulse
response as well as the locations and amplitudes of the pulses. The algorithm
alternatively estimates the pulse locations and the pulse shape in a manner
reminiscent of classical deconvolution algorithms. Numerical experiments on
synthetic and real data demonstrate the advantages of our approach over
standard CS
FroDO: From Detections to 3D Objects
Object-oriented maps are important for scene understanding since they jointly capture geometry and semantics, allow individual instantiation and meaningful reasoning about objects. We introduce FroDO, a method for accurate 3D reconstruction of object instances from RGB video that infers their location, pose and shape in a coarse to fine manner. Key to FroDO is to embed object shapes in a novel learnt shape space that allows seamless switching between sparse point cloud and dense DeepSDF decoding. Given an input sequence of localized RGB frames, FroDO first aggregates 2D detections to instantiate a 3D bounding box per object. A shape code is regressed using an encoder network before optimizing shape and pose further under the learnt shape priors using sparse or dense shape representations. The optimization uses multi-view geometric, photometric and silhouette losses. We evaluate on real-world datasets, including Pix3D, Redwood-OS, and ScanNet, for single-view, multi-view, and multi-object reconstruction
Recommended from our members
TVL<sub>1</sub>shape approximation from scattered 3D data
With the emergence in 3D sensors such as laser scanners and 3D reconstruction from cameras, large 3D point clouds can now be sampled from physical objects within a scene. The raw 3D samples delivered by these sensors however, contain only a limited degree of information about the environment the objects exist in, which means that further geometrical high-level modelling is essential. In addition, issues like sparse data measurements, noise, missing samples due to occlusion, and the inherently huge datasets involved in such representations makes this task extremely challenging. This paper addresses these issues by presenting a new 3D shape modelling framework for samples acquired from 3D sensor. Motivated by the success of nonlinear kernel-based approximation techniques in the statistics domain, existing methods using radial basis functions are applied to 3D object shape approximation. The task is framed as an optimization problem and is extended using non-smooth L1 total variation regularization. Appropriate convex energy functionals are constructed and solved by applying the Alternating Direction Method of Multipliers approach, which is then extended using Gauss-Seidel iterations. This significantly lowers the computational complexity involved in generating 3D shape from 3D samples, while both numerical and qualitative analysis confirms the superior shape modelling performance of this new framework compared with existing 3D shape reconstruction techniques
Linear Shape Deformation Models with Local Support Using Graph-based Structured Matrix Factorisation
Representing 3D shape deformations by linear models in high-dimensional space
has many applications in computer vision and medical imaging, such as
shape-based interpolation or segmentation. Commonly, using Principal Components
Analysis a low-dimensional (affine) subspace of the high-dimensional shape
space is determined. However, the resulting factors (the most dominant
eigenvectors of the covariance matrix) have global support, i.e. changing the
coefficient of a single factor deforms the entire shape. In this paper, a
method to obtain deformation factors with local support is presented. The
benefits of such models include better flexibility and interpretability as well
as the possibility of interactively deforming shapes locally. For that, based
on a well-grounded theoretical motivation, we formulate a matrix factorisation
problem employing sparsity and graph-based regularisation terms. We demonstrate
that for brain shapes our method outperforms the state of the art in local
support models with respect to generalisation ability and sparse shape
reconstruction, whereas for human body shapes our method gives more realistic
deformations.Comment: Please cite CVPR 2016 versio
- …