183 research outputs found
Fast and Accurate Depth Estimation from Sparse Light Fields
We present a fast and accurate method for dense depth reconstruction from
sparsely sampled light fields obtained using a synchronized camera array. In
our method, the source images are over-segmented into non-overlapping compact
superpixels that are used as basic data units for depth estimation and
refinement. Superpixel representation provides a desirable reduction in the
computational cost while preserving the image geometry with respect to the
object contours. Each superpixel is modeled as a plane in the image space,
allowing depth values to vary smoothly within the superpixel area. Initial
depth maps, which are obtained by plane sweeping, are iteratively refined by
propagating good correspondences within an image. To ensure the fast
convergence of the iterative optimization process, we employ a highly parallel
propagation scheme that operates on all the superpixels of all the images at
once, making full use of the parallel graphics hardware. A few optimization
iterations of the energy function incorporating superpixel-wise smoothness and
geometric consistency constraints allows to recover depth with high accuracy in
textured and textureless regions as well as areas with occlusions, producing
dense globally consistent depth maps. We demonstrate that while the depth
reconstruction takes about a second per full high-definition view, the accuracy
of the obtained depth maps is comparable with the state-of-the-art results.Comment: 15 pages, 15 figure
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
We present a neural rendering-based method called NeRO for reconstructing the
geometry and the BRDF of reflective objects from multiview images captured in
an unknown environment. Multiview reconstruction of reflective objects is
extremely challenging because specular reflections are view-dependent and thus
violate the multiview consistency, which is the cornerstone for most multiview
reconstruction methods. Recent neural rendering techniques can model the
interaction between environment lights and the object surfaces to fit the
view-dependent reflections, thus making it possible to reconstruct reflective
objects from multiview images. However, accurately modeling environment lights
in the neural rendering is intractable, especially when the geometry is
unknown. Most existing neural rendering methods, which can model environment
lights, only consider direct lights and rely on object masks to reconstruct
objects with weak specular reflections. Therefore, these methods fail to
reconstruct reflective objects, especially when the object mask is not
available and the object is illuminated by indirect lights. We propose a
two-step approach to tackle this problem. First, by applying the split-sum
approximation and the integrated directional encoding to approximate the
shading effects of both direct and indirect lights, we are able to accurately
reconstruct the geometry of reflective objects without any object masks. Then,
with the object geometry fixed, we use more accurate sampling to recover the
environment lights and the BRDF of the object. Extensive experiments
demonstrate that our method is capable of accurately reconstructing the
geometry and the BRDF of reflective objects from only posed RGB images without
knowing the environment lights and the object masks. Codes and datasets are
available at https://github.com/liuyuan-pal/NeRO.Comment: Accepted to SIGGRAPH 2023. Project page:
https://liuyuan-pal.github.io/NeRO/ Codes:
https://github.com/liuyuan-pal/NeR
V-FUSE: Volumetric Depth Map Fusion with Long-Range Constraints
We introduce a learning-based depth map fusion framework that accepts a set
of depth and confidence maps generated by a Multi-View Stereo (MVS) algorithm
as input and improves them. This is accomplished by integrating volumetric
visibility constraints that encode long-range surface relationships across
different views into an end-to-end trainable architecture. We also introduce a
depth search window estimation sub-network trained jointly with the larger
fusion sub-network to reduce the depth hypothesis search space along each ray.
Our method learns to model depth consensus and violations of visibility
constraints directly from the data; effectively removing the necessity of
fine-tuning fusion parameters. Extensive experiments on MVS datasets show
substantial improvements in the accuracy of the output fused depth and
confidence maps.Comment: ICCV 202
- …