2 research outputs found
Pro-Cam SSfM: Projector-Camera System for Structure and Spectral Reflectance from Motion
In this paper, we propose a novel projector-camera system for practical and
low-cost acquisition of a dense object 3D model with the spectral reflectance
property. In our system, we use a standard RGB camera and leverage an
off-the-shelf projector as active illumination for both the 3D reconstruction
and the spectral reflectance estimation. We first reconstruct the 3D points
while estimating the poses of the camera and the projector, which are
alternately moved around the object, by combining multi-view structured light
and structure-from-motion (SfM) techniques. We then exploit the projector for
multispectral imaging and estimate the spectral reflectance of each 3D point
based on a novel spectral reflectance estimation model considering the
geometric relationship between the reconstructed 3D points and the estimated
projector positions. Experimental results on several real objects demonstrate
that our system can precisely acquire a dense 3D model with the full spectral
reflectance property using off-the-shelf devices.Comment: Accepted by ICCV 2019. Project homepage:
http://www.ok.sc.e.titech.ac.jp/res/PCSSfM
End-to-End Hyperspectral-Depth Imaging with Learned Diffractive Optics
To extend the capabilities of spectral imaging, hyperspectral and depth
imaging have been combined to capture the higher-dimensional visual
information. However, the form factor of the combined imaging systems
increases, limiting the applicability of this new technology. In this work, we
propose a monocular imaging system for simultaneously capturing
hyperspectral-depth (HS-D) scene information with an optimized diffractive
optical element (DOE). In the training phase, this DOE is optimized jointly
with a convolutional neural network to estimate HS-D data from a snapshot
input. To study natural image statistics of this high-dimensional visual data
and to enable such a machine learning-based DOE training procedure, we record
two HS-D datasets. One is used for end-to-end optimization in deep optical HS-D
imaging, and the other is used for enhancing reconstruction performance with a
real-DOE prototype. The optimized DOE is fabricated with a grayscale
lithography process and inserted into a portable HS-D camera prototype, which
is shown to robustly capture HS-D information. In extensive evaluations, we
demonstrate that our deep optical imaging system achieves state-of-the-art
results for HS-D imaging and that the optimized DOE outperforms alternative
optical designs