1,573 research outputs found
3D Scene Geometry Estimation from 360 Imagery: A Survey
This paper provides a comprehensive survey on pioneer and state-of-the-art 3D
scene geometry estimation methodologies based on single, two, or multiple
images captured under the omnidirectional optics. We first revisit the basic
concepts of the spherical camera model, and review the most common acquisition
technologies and representation formats suitable for omnidirectional (also
called 360, spherical or panoramic) images and videos. We then survey
monocular layout and depth inference approaches, highlighting the recent
advances in learning-based solutions suited for spherical data. The classical
stereo matching is then revised on the spherical domain, where methodologies
for detecting and describing sparse and dense features become crucial. The
stereo matching concepts are then extrapolated for multiple view camera setups,
categorizing them among light fields, multi-view stereo, and structure from
motion (or visual simultaneous localization and mapping). We also compile and
discuss commonly adopted datasets and figures of merit indicated for each
purpose and list recent results for completeness. We conclude this paper by
pointing out current and future trends.Comment: Published in ACM Computing Survey
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
We introduce a method to convert stereo 360{\deg} (omnidirectional stereo)
imagery into a layered, multi-sphere image representation for six
degree-of-freedom (6DoF) rendering. Stereo 360{\deg} imagery can be captured
from multi-camera systems for virtual reality (VR), but lacks motion parallax
and correct-in-all-directions disparity cues. Together, these can quickly lead
to VR sickness when viewing content. One solution is to try and generate a
format suitable for 6DoF rendering, such as by estimating depth. However, this
raises questions as to how to handle disoccluded regions in dynamic scenes. Our
approach is to simultaneously learn depth and disocclusions via a multi-sphere
image representation, which can be rendered with correct 6DoF disparity and
motion parallax in VR. This significantly improves comfort for the viewer, and
can be inferred and rendered in real time on modern GPU hardware. Together,
these move towards making VR video a more comfortable immersive medium.Comment: 25 pages, 13 figures, Published at European Conference on Computer
Vision (ECCV 2020), Project Page: http://visual.cs.brown.edu/matryodshk
- …