927 research outputs found
Second-order Shape Optimization for Geometric Inverse Problems in Vision
We develop a method for optimization in shape spaces, i.e., sets of surfaces
modulo re-parametrization. Unlike previously proposed gradient flows, we
achieve superlinear convergence rates through a subtle approximation of the
shape Hessian, which is generally hard to compute and suffers from a series of
degeneracies. Our analysis highlights the role of mean curvature motion in
comparison with first-order schemes: instead of surface area, our approach
penalizes deformation, either by its Dirichlet energy or total variation.
Latter regularizer sparks the development of an alternating direction method of
multipliers on triangular meshes. Therein, a conjugate-gradients solver enables
us to bypass formation of the Gaussian normal equations appearing in the course
of the overall optimization. We combine all of the aforementioned ideas in a
versatile geometric variation-regularized Levenberg-Marquardt-type method
applicable to a variety of shape functionals, depending on intrinsic properties
of the surface such as normal field and curvature as well as its embedding into
space. Promising experimental results are reported
Semantically Informed Multiview Surface Refinement
We present a method to jointly refine the geometry and semantic segmentation
of 3D surface meshes. Our method alternates between updating the shape and the
semantic labels. In the geometry refinement step, the mesh is deformed with
variational energy minimization, such that it simultaneously maximizes
photo-consistency and the compatibility of the semantic segmentations across a
set of calibrated images. Label-specific shape priors account for interactions
between the geometry and the semantic labels in 3D. In the semantic
segmentation step, the labels on the mesh are updated with MRF inference, such
that they are compatible with the semantic segmentations in the input images.
Also, this step includes prior assumptions about the surface shape of different
semantic classes. The priors induce a tight coupling, where semantic
information influences the shape update and vice versa. Specifically, we
introduce priors that favor (i) adaptive smoothing, depending on the class
label; (ii) straightness of class boundaries; and (iii) semantic labels that
are consistent with the surface orientation. The novel mesh-based
reconstruction is evaluated in a series of experiments with real and synthetic
data. We compare both to state-of-the-art, voxel-based semantic 3D
reconstruction, and to purely geometric mesh refinement, and demonstrate that
the proposed scheme yields improved 3D geometry as well as an improved semantic
segmentation
Contour Generator Points for Threshold Selection and a Novel Photo-Consistency Measure for Space Carving
Space carving has emerged as a powerful method for multiview scene reconstruction. Although a wide variety of methods have been proposed, the quality of the reconstruction remains highly-dependent on the photometric consistency measure, and the threshold used to carve away voxels. In this paper, we present a novel photo-consistency measure that is motivated by a multiset variant of the chamfer distance. The new measure is robust to high amounts of within-view color variance and also takes into account the projection angles of back-projected pixels.
Another critical issue in space carving is the selection of the photo-consistency threshold used to determine what surface voxels are kept or carved away. In this paper, a reliable threshold selection technique is proposed that examines the photo-consistency values at contour generator points. Contour generators are points that lie on both the surface of the object and the visual hull. To determine the threshold, a percentile ranking of the photo-consistency values of these generator points is used. This improved technique is applicable to a wide variety of photo-consistency measures, including the new measure presented in this paper. Also presented in this paper is a method to choose between photo-consistency measures, and voxel array resolutions prior to carving using receiver operating characteristic (ROC) curves
From small to large baseline multiview stereo : dealing with blur, clutter and occlusions
This thesis addresses the problem of reconstructing the three-dimensional
(3D) digital model of a scene from a collection of two-dimensional (2D)
images taken from it. To address this fundamental computer vision
problem, we propose three algorithms. They are the main contributions
of this thesis.
First, we solve multiview stereo with the o -axis aperture camera.
This system has a very small baseline as images are captured from
viewpoints close to each other. The key idea is to change the size or
the 3D location of the aperture of the camera so as to extract selected
portions of the scene. Our imaging model takes both defocus and
stereo information into account and allows to solve shape reconstruction
and image restoration in one go. The o -axis aperture camera can
be used in a small-scale space where the camera motion is constrained
by the surrounding environment, such as in 3D endoscopy.
Second, to solve multiview stereo with large baseline, we present a
framework that poses the problem of recovering a 3D surface in the
scene as a regularized minimal partition problem of a visibility function.
The formulation is convex and hence guarantees that the solution
converges to the global minimum. Our formulation is robust
to view-varying extensive occlusions, clutter and image noise. At
any stage during the estimation process the method does not rely on
the visual hull, 2D silhouettes, approximate depth maps, or knowing
which views are dependent(i.e., overlapping) and which are independent(
i.e., non overlapping). Furthermore, the degenerate solution, the
null surface, is not included as a global solution in this formulation.
One limitation of this algorithm is that its computation complexity
grows with the number of views that we combine simultaneously. To
address this limitation, we propose a third formulation. In this formulation,
the visibility functions are integrated within a narrow band
around the estimated surface by setting weights to each point along
optical rays.
This thesis presents technical descriptions for each algorithm and detailed
analyses to show how these algorithms improve existing reconstruction
techniques
Towards high-resolution large-scale multi-view stereo
International audienceBoosted by the Middlebury challenge, the precision of dense multi-view stereovision methods has increased drastically in the past few years. Yet, most methods, although they perform well on this benchmark, are still inapplicable to large-scale data sets taken under uncontrolled conditions. In this paper, we propose a multi-view stereo pipeline able to deal at the same time with very large scenes while still producing highly detailed reconstructions within very reasonable time. The keys to these benefits are twofold: (i) a minimum s-t cut based global optimization that transforms a dense point cloud into a visibility consistent mesh, followed by (ii) a mesh-based variational refinement that captures small details, smartly handling photo-consistency, regularization and adaptive resolution. Our method has been tested on numerous large-scale outdoor scenes. The accuracy of our reconstructions is also measured on the recent dense multi-view benchmark proposed by Strecha et al., showing our results to compare more than favorably with the current state-of-the-art
- …