371 research outputs found
PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas
Achieving an immersive experience enabling users to explore virtual
environments with six degrees of freedom (6DoF) is essential for various
applications such as virtual reality (VR). Wide-baseline panoramas are commonly
used in these applications to reduce network bandwidth and storage
requirements. However, synthesizing novel views from these panoramas remains a
key challenge. Although existing neural radiance field methods can produce
photorealistic views under narrow-baseline and dense image captures, they tend
to overfit the training views when dealing with \emph{wide-baseline} panoramas
due to the difficulty in learning accurate geometry from sparse
views. To address this problem, we propose PanoGRF, Generalizable Spherical
Radiance Fields for Wide-baseline Panoramas, which construct spherical radiance
fields incorporating scene priors. Unlike generalizable radiance
fields trained on perspective images, PanoGRF avoids the information loss from
panorama-to-perspective conversion and directly aggregates geometry and
appearance features of 3D sample points from each panoramic view based on
spherical projection. Moreover, as some regions of the panorama are only
visible from one view while invisible from others under wide baseline settings,
PanoGRF incorporates monocular depth priors into spherical depth
estimation to improve the geometry features. Experimental results on multiple
panoramic datasets demonstrate that PanoGRF significantly outperforms
state-of-the-art generalizable view synthesis methods for wide-baseline
panoramas (e.g., OmniSyn) and perspective images (e.g., IBRNet, NeuRay)
Walk2Map: Extracting Floor Plans from Indoor Walk Trajectories
Recent years have seen a proliferation of new digital products for the
efficient management of indoor spaces, with important applications like
emergency management, virtual property showcasing and interior design. These
products rely on accurate 3D models of the environments considered, including
information on both architectural and non-permanent elements. These models must
be created from measured data such as RGB-D images or 3D point clouds, whose
capture and consolidation involves lengthy data workflows. This strongly limits
the rate at which 3D models can be produced, preventing the adoption of many
digital services for indoor space management. We provide an alternative to such
data-intensive procedures by presenting Walk2Map, a data-driven approach to
generate floor plans only from trajectories of a person walking inside the
rooms. Thanks to recent advances in data-driven inertial odometry, such
minimalistic input data can be acquired from the IMU readings of consumer-level
smartphones, which allows for an effortless and scalable mapping of real-world
indoor spaces. Our work is based on learning the latent relation between an
indoor walk trajectory and the information represented in a floor plan:
interior space footprint, portals, and furniture. We distinguish between
recovering area-related (interior footprint, furniture) and wall-related
(doors) information and use two different neural architectures for the two
tasks: an image-based Encoder-Decoder and a Graph Convolutional Network,
respectively. We train our networks using scanned 3D indoor models and apply
them in a cascaded fashion on an indoor walk trajectory at inference time. We
perform a qualitative and quantitative evaluation using both simulated and
measured, real-world trajectories, and compare against a baseline method for
image-to-image translation. The experiments confirm the feasibility of our
approach.Comment: To be published in Computer Graphics Forum (Proc. Eurographics 2021
Recommended from our members
CLOI: A Shape Classification Benchmark Dataset for Industrial Facilities
Generation of digital models of existing industrial facilities is labor intensive and expensive. The use of state-of-the-art deep learning algorithms can assist to reduce the modelling time and cost. However large databases of labelled, laser-scanned industrial facilities do not exist to date, henceforth training of deep learning models is not possible. Our paper solves this problem by proposing a new benchmark dataset, which consists of five labelled industrial plants. The labelling schema that we followed for the generation of this dataset is based on the frequency of appearance of industrial object types. We labelled the ten most frequent industrial object shapes as identified in previous work. We present CLOI (Channels, L-shapes, circular sections, I-shapes): a richly annotated large-scale repository of shapes represented by labelled point clusters. CLOI has more than 140 million hand labelled points and serves as the foundation for researchers who are interested in automated modelling of industrial assets using deep learning algorithms
- …