288 research outputs found
DCTM: Discrete-Continuous Transformation Matching for Semantic Flow
Techniques for dense semantic correspondence have provided limited ability to
deal with the geometric variations that commonly exist between semantically
similar images. While variations due to scale and rotation have been examined,
there lack practical solutions for more complex deformations such as affine
transformations because of the tremendous size of the associated solution
space. To address this problem, we present a discrete-continuous transformation
matching (DCTM) framework where dense affine transformation fields are inferred
through a discrete label optimization in which the labels are iteratively
updated via continuous regularization. In this way, our approach draws
solutions from the continuous space of affine transformations in a manner that
can be computed efficiently through constant-time edge-aware filtering and a
proposed affine-varying CNN-based descriptor. Experimental results show that
this model outperforms the state-of-the-art methods for dense semantic
correspondence on various benchmarks
MP-MVS: Multi-Scale Windows PatchMatch and Planar Prior Multi-View Stereo
Significant strides have been made in enhancing the accuracy of Multi-View
Stereo (MVS)-based 3D reconstruction. However, untextured areas with unstable
photometric consistency often remain incompletely reconstructed. In this paper,
we propose a resilient and effective multi-view stereo approach (MP-MVS). We
design a multi-scale windows PatchMatch (mPM) to obtain reliable depth of
untextured areas. In contrast with other multi-scale approaches, which is
faster and can be easily extended to PatchMatch-based MVS approaches.
Subsequently, we improve the existing checkerboard sampling schemes by limiting
our sampling to distant regions, which can effectively improve the efficiency
of spatial propagation while mitigating outlier generation. Finally, we
introduce and improve planar prior assisted PatchMatch of ACMP. Instead of
relying on photometric consistency, we utilize geometric consistency
information between multi-views to select reliable triangulated vertices. This
strategy can obtain a more accurate planar prior model to rectify photometric
consistency measurements. Our approach has been tested on the ETH3D High-res
multi-view benchmark with several state-of-the-art approaches. The results
demonstrate that our approach can reach the state-of-the-art. The associated
codes will be accessible at https://github.com/RongxuanTan/MP-MVS
Semantically Derived Geometric Constraints for {MVS} Reconstruction of Textureless Areas
Conventional multi-view stereo (MVS) approaches based on photo-consistency measures are generally robust, yet often fail in calculating valid depth pixel estimates in low textured areas of the scene. In this study, a novel approach is proposed to tackle this challenge by leveraging semantic priors into a PatchMatch-based MVS in order to increase confidence and support depth and normal map estimation. Semantic class labels on image pixels are used to impose class-specific geometric constraints during multiview stereo, optimising the depth estimation on weakly supported, textureless areas, commonly present in urban scenarios of building facades, indoor scenes, or aerial datasets. Detecting dominant shapes, e.g., planes, with RANSAC, an adjusted cost function is introduced that combines and weighs both photometric and semantic scores propagating, thus, more accurate depth estimates. Being adaptive, it fills in apparent information gaps and smoothing local roughness in problematic regions while at the same time preserves important details. Experiments on benchmark and custom datasets demonstrate the effectiveness of the presented approach
Polarimetric PatchMatch Multi-View Stereo
PatchMatch Multi-View Stereo (PatchMatch MVS) is one of the popular MVS
approaches, owing to its balanced accuracy and efficiency. In this paper, we
propose Polarimetric PatchMatch multi-view Stereo (PolarPMS), which is the
first method exploiting polarization cues to PatchMatch MVS. The key of
PatchMatch MVS is to generate depth and normal hypotheses, which form local 3D
planes and slanted stereo matching windows, and efficiently search for the best
hypothesis based on the consistency among multi-view images. In addition to
standard photometric consistency, our PolarPMS evaluates polarimetric
consistency to assess the validness of a depth and normal hypothesis, motivated
by the physical property that the polarimetric information is related to the
object's surface normal. Experimental results demonstrate that our PolarPMS can
improve the accuracy and the completeness of reconstructed 3D models,
especially for texture-less surfaces, compared with state-of-the-art PatchMatch
MVS methods
ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems
In this paper we present ActiveStereoNet, the first deep learning solution
for active stereo systems. Due to the lack of ground truth, our method is fully
self-supervised, yet it produces precise depth with a subpixel precision of
of a pixel; it does not suffer from the common over-smoothing issues;
it preserves the edges; and it explicitly handles occlusions. We introduce a
novel reconstruction loss that is more robust to noise and texture-less
patches, and is invariant to illumination changes. The proposed loss is
optimized using a window-based cost aggregation with an adaptive support weight
scheme. This cost aggregation is edge-preserving and smooths the loss function,
which is key to allow the network to reach compelling results. Finally we show
how the task of predicting invalid regions, such as occlusions, can be trained
end-to-end without ground-truth. This component is crucial to reduce blur and
particularly improves predictions along depth discontinuities. Extensive
quantitatively and qualitatively evaluations on real and synthetic data
demonstrate state of the art results in many challenging scenes.Comment: Accepted by ECCV2018, Oral Presentation, Main paper + Supplementary
Material
- …