274 research outputs found
Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis
We introduce a data-driven approach to complete partial 3D shapes through a
combination of volumetric deep neural networks and 3D shape synthesis. From a
partially-scanned input shape, our method first infers a low-resolution -- but
complete -- output. To this end, we introduce a 3D-Encoder-Predictor Network
(3D-EPN) which is composed of 3D convolutional layers. The network is trained
to predict and fill in missing data, and operates on an implicit surface
representation that encodes both known and unknown space. This allows us to
predict global structure in unknown areas at high accuracy. We then correlate
these intermediary results with 3D geometry from a shape database at test time.
In a final pass, we propose a patch-based 3D shape synthesis method that
imposes the 3D geometry from these retrieved shapes as constraints on the
coarsely-completed mesh. This synthesis process enables us to reconstruct
fine-scale detail and generate high-resolution output while respecting the
global mesh structure obtained by the 3D-EPN. Although our 3D-EPN outperforms
state-of-the-art completion method, the main contribution in our work lies in
the combination of a data-driven shape predictor and analytic 3D shape
synthesis. In our results, we show extensive evaluations on a newly-introduced
shape completion benchmark for both real-world and synthetic data
End2End Multi-View Feature Matching with Differentiable Pose Optimization
Erroneous feature matches have severe impact on subsequent camera pose
estimation and often require additional, time-costly measures, like RANSAC, for
outlier rejection. Our method tackles this challenge by addressing feature
matching and pose optimization jointly. To this end, we propose a graph
attention network to predict image correspondences along with confidence
weights. The resulting matches serve as weighted constraints in a
differentiable pose estimation. Training feature matching with gradients from
pose optimization naturally learns to down-weight outliers and boosts pose
estimation on image pairs compared to SuperGlue by 6.7% on ScanNet. At the same
time, it reduces the pose estimation time by over 50% and renders RANSAC
iterations unnecessary. Moreover, we integrate information from multiple views
by spanning the graph across multiple frames to predict the matches all at
once. Multi-view matching combined with end-to-end training improves the pose
estimation metrics on Matterport3D by 18.5% compared to SuperGlue.Comment: ICCV 2023, project page:
https://barbararoessle.github.io/e2e_multi_view_matching , video:
https://youtu.be/uuLb6GfM9C
- …