3,594 research outputs found
Deep Learning Methods for Calibrated Photometric Stereo and Beyond
Photometric stereo recovers the surface normals of an object from multiple
images with varying shading cues, i.e., modeling the relationship between
surface orientation and intensity at each pixel. Photometric stereo prevails in
superior per-pixel resolution and fine reconstruction details. However, it is a
complicated problem because of the non-linear relationship caused by
non-Lambertian surface reflectance. Recently, various deep learning methods
have shown a powerful ability in the context of photometric stereo against
non-Lambertian surfaces. This paper provides a comprehensive review of existing
deep learning-based calibrated photometric stereo methods. We first analyze
these methods from different perspectives, including input processing,
supervision, and network architecture. We summarize the performance of deep
learning photometric stereo models on the most widely-used benchmark data set.
This demonstrates the advanced performance of deep learning-based photometric
stereo methods. Finally, we give suggestions and propose future research trends
based on the limitations of existing models.Comment: 19 pages, 11 figures, 4 table
Photometric Depth Super-Resolution
This study explores the use of photometric techniques (shape-from-shading and
uncalibrated photometric stereo) for upsampling the low-resolution depth map
from an RGB-D sensor to the higher resolution of the companion RGB image. A
single-shot variational approach is first put forward, which is effective as
long as the target's reflectance is piecewise-constant. It is then shown that
this dependency upon a specific reflectance model can be relaxed by focusing on
a specific class of objects (e.g., faces), and delegate reflectance estimation
to a deep neural network. A multi-shot strategy based on randomly varying
lighting conditions is eventually discussed. It requires no training or prior
on the reflectance, yet this comes at the price of a dedicated acquisition
setup. Both quantitative and qualitative evaluations illustrate the
effectiveness of the proposed methods on synthetic and real-world scenarios.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence
(T-PAMI), 2019. First three authors contribute equall
PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism
Existing deep calibrated photometric stereo networks basically aggregate
observations under different lights based on the pre-defined operations such as
linear projection and max pooling. While they are effective with the dense
capture, simple first-order operations often fail to capture the high-order
interactions among observations under small number of different lights. To
tackle this issue, this paper presents a deep sparse calibrated photometric
stereo network named {\it PS-Transformer} which leverages the learnable
self-attention mechanism to properly capture the complex inter-image
interactions. PS-Transformer builds upon the dual-branch design to explore both
pixel-wise and image-wise features and individual feature is trained with the
intermediate surface normal supervision to maximize geometric feasibility. A
new synthetic dataset named CyclesPS+ is also presented with the comprehensive
analysis to successfully train the photometric stereo networks. Extensive
results on the publicly available benchmark datasets demonstrate that the
surface normal prediction accuracy of the proposed method significantly
outperforms other state-of-the-art algorithms with the same number of input
images and is even comparable to that of dense algorithms which input
10 larger number of images.Comment: BMVC2021. Code and Supplementary are available at
https://github.com/satoshi-ikehata/PS-Transformer-BMVC202
- …