97 research outputs found
MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures
Neural Radiance Fields (NeRFs) have demonstrated amazing ability to
synthesize images of 3D scenes from novel views. However, they rely upon
specialized volumetric rendering algorithms based on ray marching that are
mismatched to the capabilities of widely deployed graphics hardware. This paper
introduces a new NeRF representation based on textured polygons that can
synthesize novel images efficiently with standard rendering pipelines. The NeRF
is represented as a set of polygons with textures representing binary opacities
and feature vectors. Traditional rendering of the polygons with a z-buffer
yields an image with features at every pixel, which are interpreted by a small,
view-dependent MLP running in a fragment shader to produce a final pixel color.
This approach enables NeRFs to be rendered with the traditional polygon
rasterization pipeline, which provides massive pixel-level parallelism,
achieving interactive frame rates on a wide range of compute platforms,
including mobile phones.Comment: CVPR 2023. Project page: https://mobile-nerf.github.io, code:
https://github.com/google-research/jax3d/tree/main/jax3d/projects/mobilener
nerf2nerf: Pairwise Registration of Neural Radiance Fields
We introduce a technique for pairwise registration of neural fields that
extends classical optimization-based local registration (i.e. ICP) to operate
on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained
from collections of calibrated images. NeRF does not decompose illumination and
color, so to make registration invariant to illumination, we introduce the
concept of a ''surface field'' -- a field distilled from a pre-trained NeRF
model that measures the likelihood of a point being on the surface of an
object. We then cast nerf2nerf registration as a robust optimization that
iteratively seeks a rigid transformation that aligns the surface fields of the
two scenes. We evaluate the effectiveness of our technique by introducing a
dataset of pre-trained NeRF scenes -- our synthetic scenes enable quantitative
evaluations and comparisons to classical registration techniques, while our
real scenes demonstrate the validity of our technique in real-world scenarios.
Additional results available at: https://nerf2nerf.github.i
- …