145 research outputs found
Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation
Recent studies have witnessed that self-supervised methods based on view
synthesis obtain clear progress on multi-view stereo (MVS). However, existing
methods rely on the assumption that the corresponding points among different
views share the same color, which may not always be true in practice. This may
lead to unreliable self-supervised signal and harm the final reconstruction
performance. To address the issue, we propose a framework integrated with more
reliable supervision guided by semantic co-segmentation and data-augmentation.
Specially, we excavate mutual semantic from multi-view images to guide the
semantic consistency. And we devise effective data-augmentation mechanism which
ensures the transformation robustness by treating the prediction of regular
samples as pseudo ground truth to regularize the prediction of augmented
samples. Experimental results on DTU dataset show that our proposed methods
achieve the state-of-the-art performance among unsupervised methods, and even
compete on par with supervised methods. Furthermore, extensive experiments on
Tanks&Temples dataset demonstrate the effective generalization ability of the
proposed method.Comment: This paper is accepted by AAAI-21 with a Distinguished Paper Awar
Semi-supervised Deep Multi-view Stereo
Significant progress has been witnessed in learning-based Multi-view Stereo
(MVS) under supervised and unsupervised settings. To combine their respective
merits in accuracy and completeness, meantime reducing the demand for expensive
labeled data, this paper explores the problem of learning-based MVS in a
semi-supervised setting that only a tiny part of the MVS data is attached with
dense depth ground truth. However, due to huge variation of scenarios and
flexible settings in views, it may break the basic assumption in classic
semi-supervised learning, that unlabeled data and labeled data share the same
label space and data distribution, named as semi-supervised distribution-gap
ambiguity in the MVS problem. To handle these issues, we propose a novel
semi-supervised distribution-augmented MVS framework, namely SDA-MVS. For the
simple case that the basic assumption works in MVS data, consistency
regularization encourages the model predictions to be consistent between
original sample and randomly augmented sample. For further troublesome case
that the basic assumption is conflicted in MVS data, we propose a novel style
consistency loss to alleviate the negative effect caused by the distribution
gap. The visual style of unlabeled sample is transferred to labeled sample to
shrink the gap, and the model prediction of generated sample is further
supervised with the label in original labeled sample. The experimental results
in semi-supervised settings of multiple MVS datasets show the superior
performance of the proposed method. With the same settings in backbone network,
our proposed SDA-MVS outperforms its fully-supervised and unsupervised
baselines.Comment: This paper is accepted in ACMMM-2023. The code is released at:
https://github.com/ToughStoneX/Semi-MV
- …