5,061 research outputs found
Planar Prior Assisted PatchMatch Multi-View Stereo
The completeness of 3D models is still a challenging problem in multi-view
stereo (MVS) due to the unreliable photometric consistency in low-textured
areas. Since low-textured areas usually exhibit strong planarity, planar models
are advantageous to the depth estimation of low-textured areas. On the other
hand, PatchMatch multi-view stereo is very efficient for its sampling and
propagation scheme. By taking advantage of planar models and PatchMatch
multi-view stereo, we propose a planar prior assisted PatchMatch multi-view
stereo framework in this paper. In detail, we utilize a probabilistic graphical
model to embed planar models into PatchMatch multi-view stereo and contribute a
novel multi-view aggregated matching cost. This novel cost takes both
photometric consistency and planar compatibility into consideration, making it
suited for the depth estimation of both non-planar and planar regions.
Experimental results demonstrate that our method can efficiently recover the
depth information of extremely low-textured areas, thus obtaining high complete
3D models and achieving state-of-the-art performance.Comment: Accepted by AAAI-202
Multi-View Stereo with Single-View Semantic Mesh Refinement
While 3D reconstruction is a well-established and widely explored research
topic, semantic 3D reconstruction has only recently witnessed an increasing
share of attention from the Computer Vision community. Semantic annotations
allow in fact to enforce strong class-dependent priors, as planarity for ground
and walls, which can be exploited to refine the reconstruction often resulting
in non-trivial performance improvements. State-of-the art methods propose
volumetric approaches to fuse RGB image data with semantic labels; even if
successful, they do not scale well and fail to output high resolution meshes.
In this paper we propose a novel method to refine both the geometry and the
semantic labeling of a given mesh. We refine the mesh geometry by applying a
variational method that optimizes a composite energy made of a state-of-the-art
pairwise photo-metric term and a single-view term that models the semantic
consistency between the labels of the 3D mesh and those of the segmented
images. We also update the semantic labeling through a novel Markov Random
Field (MRF) formulation that, together with the classical data and smoothness
terms, takes into account class-specific priors estimated directly from the
annotated mesh. This is in contrast to state-of-the-art methods that are
typically based on handcrafted or learned priors. We are the first, jointly
with the very recent and seminal work of [M. Blaha et al arXiv:1706.08336,
2017], to propose the use of semantics inside a mesh refinement framework.
Differently from [M. Blaha et al arXiv:1706.08336, 2017], which adopts a more
classical pairwise comparison to estimate the flow of the mesh, we apply a
single-view comparison between the semantically annotated image and the current
3D mesh labels; this improves the robustness in case of noisy segmentations.Comment: {\pounds}D Reconstruction Meets Semantic, ICCV worksho
Polarimetric PatchMatch Multi-View Stereo
PatchMatch Multi-View Stereo (PatchMatch MVS) is one of the popular MVS
approaches, owing to its balanced accuracy and efficiency. In this paper, we
propose Polarimetric PatchMatch multi-view Stereo (PolarPMS), which is the
first method exploiting polarization cues to PatchMatch MVS. The key of
PatchMatch MVS is to generate depth and normal hypotheses, which form local 3D
planes and slanted stereo matching windows, and efficiently search for the best
hypothesis based on the consistency among multi-view images. In addition to
standard photometric consistency, our PolarPMS evaluates polarimetric
consistency to assess the validness of a depth and normal hypothesis, motivated
by the physical property that the polarimetric information is related to the
object's surface normal. Experimental results demonstrate that our PolarPMS can
improve the accuracy and the completeness of reconstructed 3D models,
especially for texture-less surfaces, compared with state-of-the-art PatchMatch
MVS methods
Detail-preserving and Content-aware Variational Multi-view Stereo Reconstruction
Accurate recovery of 3D geometrical surfaces from calibrated 2D multi-view
images is a fundamental yet active research area in computer vision. Despite
the steady progress in multi-view stereo reconstruction, most existing methods
are still limited in recovering fine-scale details and sharp features while
suppressing noises, and may fail in reconstructing regions with few textures.
To address these limitations, this paper presents a Detail-preserving and
Content-aware Variational (DCV) multi-view stereo method, which reconstructs
the 3D surface by alternating between reprojection error minimization and mesh
denoising. In reprojection error minimization, we propose a novel inter-image
similarity measure, which is effective to preserve fine-scale details of the
reconstructed surface and builds a connection between guided image filtering
and image registration. In mesh denoising, we propose a content-aware
-minimization algorithm by adaptively estimating the value and
regularization parameters based on the current input. It is much more promising
in suppressing noise while preserving sharp features than conventional
isotropic mesh smoothing. Experimental results on benchmark datasets
demonstrate that our DCV method is capable of recovering more surface details,
and obtains cleaner and more accurate reconstructions than state-of-the-art
methods. In particular, our method achieves the best results among all
published methods on the Middlebury dino ring and dino sparse ring datasets in
terms of both completeness and accuracy.Comment: 14 pages,16 figures. Submitted to IEEE Transaction on image
processin
- …