2,282 research outputs found
Scene Prior Filtering for Depth Map Super-Resolution
Multi-modal fusion is vital to the success of super-resolution of depth maps.
However, commonly used fusion strategies, such as addition and concatenation,
fall short of effectively bridging the modal gap. As a result, guided image
filtering methods have been introduced to mitigate this issue. Nevertheless, it
is observed that their filter kernels usually encounter significant texture
interference and edge inaccuracy. To tackle these two challenges, we introduce
a Scene Prior Filtering network, SPFNet, which utilizes the priors surface
normal and semantic map from large-scale models. Specifically, we design an
All-in-one Prior Propagation that computes the similarity between multi-modal
scene priors, i.e., RGB, normal, semantic, and depth, to reduce the texture
interference. In addition, we present a One-to-one Prior Embedding that
continuously embeds each single-modal prior into depth using Mutual Guided
Filtering, further alleviating the texture interference while enhancing edges.
Our SPFNet has been extensively evaluated on both real and synthetic datasets,
achieving state-of-the-art performance.Comment: 14 page
Deep Depth Completion of a Single RGB-D Image
The goal of our work is to complete the depth channel of an RGB-D image.
Commodity-grade depth cameras often fail to sense depth for shiny, bright,
transparent, and distant surfaces. To address this problem, we train a deep
network that takes an RGB image as input and predicts dense surface normals and
occlusion boundaries. Those predictions are then combined with raw depth
observations provided by the RGB-D camera to solve for depths for all pixels,
including those missing in the original observation. This method was chosen
over others (e.g., inpainting depths directly) as the result of extensive
experiments with a new depth completion benchmark dataset, where holes are
filled in training data through the rendering of surface reconstructions
created from multiview RGB-D scans. Experiments with different network inputs,
depth representations, loss functions, optimization methods, inpainting
methods, and deep depth estimation networks show that our proposed approach
provides better depth completions than these alternatives.Comment: Accepted by CVPR2018 (Spotlight). Project webpage:
http://deepcompletion.cs.princeton.edu/ This version includes supplementary
materials which provide more implementation details, quantitative evaluation,
and qualitative results. Due to file size limit, please check project website
for high-res pape
- …