26 research outputs found
PS-Transformer: Learning Sparse Photometric Stereo Network using Self-Attention Mechanism
Existing deep calibrated photometric stereo networks basically aggregate
observations under different lights based on the pre-defined operations such as
linear projection and max pooling. While they are effective with the dense
capture, simple first-order operations often fail to capture the high-order
interactions among observations under small number of different lights. To
tackle this issue, this paper presents a deep sparse calibrated photometric
stereo network named {\it PS-Transformer} which leverages the learnable
self-attention mechanism to properly capture the complex inter-image
interactions. PS-Transformer builds upon the dual-branch design to explore both
pixel-wise and image-wise features and individual feature is trained with the
intermediate surface normal supervision to maximize geometric feasibility. A
new synthetic dataset named CyclesPS+ is also presented with the comprehensive
analysis to successfully train the photometric stereo networks. Extensive
results on the publicly available benchmark datasets demonstrate that the
surface normal prediction accuracy of the proposed method significantly
outperforms other state-of-the-art algorithms with the same number of input
images and is even comparable to that of dense algorithms which input
10 larger number of images.Comment: BMVC2021. Code and Supplementary are available at
https://github.com/satoshi-ikehata/PS-Transformer-BMVC202
Scalable, Detailed and Mask-Free Universal Photometric Stereo
In this paper, we introduce SDM-UniPS, a groundbreaking Scalable, Detailed,
Mask-free, and Universal Photometric Stereo network. Our approach can recover
astonishingly intricate surface normal maps, rivaling the quality of 3D
scanners, even when images are captured under unknown, spatially-varying
lighting conditions in uncontrolled environments. We have extended previous
universal photometric stereo networks to extract spatial-light features,
utilizing all available information in high-resolution input images and
accounting for non-local interactions among surface points. Moreover, we
present a new synthetic training dataset that encompasses a diverse range of
shapes, materials, and illumination scenarios found in real-world scenes.
Through extensive evaluation, we demonstrate that our method not only surpasses
calibrated, lighting-specific techniques on public benchmarks, but also excels
with a significantly smaller number of input images even without object masks.Comment: CVPR 2023 (Highlight). The source code will be available at
https://github.com/satoshi-ikehata/SDM-UniPS-CVPR202
Field-of-View IoU for Object Detection in 360{\deg} Images
360{\deg} cameras have gained popularity over the last few years. In this
paper, we propose two fundamental techniques -- Field-of-View IoU (FoV-IoU) and
360Augmentation for object detection in 360{\deg} images. Although most object
detection neural networks designed for the perspective images are applicable to
360{\deg} images in equirectangular projection (ERP) format, their performance
deteriorates owing to the distortion in ERP images. Our method can be readily
integrated with existing perspective object detectors and significantly
improves the performance. The FoV-IoU computes the intersection-over-union of
two Field-of-View bounding boxes in a spherical image which could be used for
training, inference, and evaluation while 360Augmentation is a data
augmentation technique specific to 360{\deg} object detection task which
randomly rotates a spherical image and solves the bias due to the
sphere-to-plane projection. We conduct extensive experiments on the 360indoor
dataset with different types of perspective object detectors and show the
consistent effectiveness of our method
制約付き回帰に基づく照度差ステレオ
学位の種別: 課程博士審査委員会委員 : (主査)東京大学准教授 山﨑 俊彦, 東京大学教授, 相澤 清晴, 東京大学教授 池内 克史, 東京大学教授 佐藤 真一, 東京大学教授 佐藤 洋一, 東京大学教授 苗村 健University of Tokyo(東京大学
Photometric Stereo Using Constrained Bivariate Regression for General Isotropic Surfaces
This paper presents a photometric stereo method that is purely pixelwise and handles general isotropic surfaces in a stable manner. Following the recently proposed sum-of-lobes representation of the isotropic reflectance function, we constructed a constrained bivariate regression problem where the regression function is approximated by smooth, bivariate Bernstein polynomials. The unknown normal vec-tor was separated from the unknown reflectance function by considering the inverse representation of the image for-mation process, and then we could accurately compute the unknown surface normals by solving a simple and ef-ficient quadratic programming problem. Extensive evalu-ations that showed the state-of-the-art performance using both synthetic and real-world images were performed. 1
Robust Photometric Stereo using Sparse Regression
This paper presents a robust photometric stereo method that effectively compensates for various non-Lambertian corruptions such as specularities, shadows, and image noise. We construct a constrained sparse regression problem that enforces both Lambertian, rank-3 structure and sparse, additive corruptions. A solution method is derived using a hierarchical Bayesian approximation to accurately estimate the surface normals while simultaneously separating the non-Lambertian corruptions. Extensive evaluations are performed that show state-of-the-art performance using both synthetic and real-world images. 1