102 research outputs found
Depth Super-Resolution Meets Uncalibrated Photometric Stereo
A novel depth super-resolution approach for RGB-D sensors is presented. It
disambiguates depth super-resolution through high-resolution photometric clues
and, symmetrically, it disambiguates uncalibrated photometric stereo through
low-resolution depth cues. To this end, an RGB-D sequence is acquired from the
same viewing angle, while illuminating the scene from various uncalibrated
directions. This sequence is handled by a variational framework which fits
high-resolution shape and reflectance, as well as lighting, to both the
low-resolution depth measurements and the high-resolution RGB ones. The key
novelty consists in a new PDE-based photometric stereo regularizer which
implicitly ensures surface regularity. This allows to carry out depth
super-resolution in a purely data-driven manner, without the need for any
ad-hoc prior or material calibration. Real-world experiments are carried out
using an out-of-the-box RGB-D sensor and a hand-held LED light source.Comment: International Conference on Computer Vision (ICCV) Workshop, 201
Intégration d'un champ de gradient rapide et robuste aux discontinuités - Application à la stéréophotométrie
National audienceNous proposons plusieurs nouvelles méthodes permettant de résoudre le problème de la reconstruction 3D d’une surface à partir de son gradient, qui soient rapides et robustes aux discontinuités de profondeur. Nous proposons de rem-placer les moindres carrés par une fonctionnelle inspirée de la méthode de restauration de Perona et Malik, et montrons comment les méthodes d’intégration existantes les plus rapides peuvent être adaptées à la minimisation de cette fonctionnelle
Reconstruction tridimensionnelle par stéréophotométrie
Cette thèse traite de la reconstruction 3D par stéréophotométrie, qui consiste à utiliser plusieurs photographies d'une scène prises sous le même angle, mais sous différents éclairages. Nous nous intéressons dans un premier temps à des techniques robustes pour l'estimation des normales à la surface, et pour leur intégration en une carte de profondeur. Nous étudions ensuite deux situations où le problème est mal posé : lorsque les éclairages sont inconnus, ou lorsque seuls deux éclairages sont utilisés. La troisième partie est consacrée à l'étude de modèles plus réalistes, à la fois en ce qui concerne les éclairages et la réflectance de la surface. Ces trois premières parties nous amènent aux limites de la formulation classique de la stéréophotométrie : nous introduisons finalement, dans la partie 4, une reformulation variationnelle et différentielle du problème qui permet de dépasser ces limites
Edge-Preserving Integration of a Normal Field: Weighted Least Squares and L1 Approaches
International audienceWe introduce several new functionals, inspired from variational image denoising models, for recovering a piecewise-smooth surface from a dense estimation of its normal field. In the weighted least-squares approach, the non-differentiable elements of the surface are a priori detected so as to weight the least-squares model. To avoid this detection step, we introduce reweighted least-squares for minimising an isotropic TV-like functional, and split-Bregman iterations for L1 minimisation
MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset
The photometric stereo (PS) problem consists in reconstructing the 3D-surface
of an object, thanks to a set of photographs taken under different lighting
directions. In this paper, we propose a multi-scale architecture for PS which,
combined with a new dataset, yields state-of-the-art results. Our proposed
architecture is flexible: it permits to consider a variable number of images as
well as variable image size without loss of performance. In addition, we define
a set of constraints to allow the generation of a relevant synthetic dataset to
train convolutional neural networks for the PS problem. Our proposed dataset is
much larger than pre-existing ones, and contains many objects with challenging
materials having anisotropic reflectance (e.g. metals, glass). We show on
publicly available benchmarks that the combination of both these contributions
drastically improves the accuracy of the estimated normal field, in comparison
with previous state-of-the-art methods
Photometric Depth Super-Resolution
This study explores the use of photometric techniques (shape-from-shading and
uncalibrated photometric stereo) for upsampling the low-resolution depth map
from an RGB-D sensor to the higher resolution of the companion RGB image. A
single-shot variational approach is first put forward, which is effective as
long as the target's reflectance is piecewise-constant. It is then shown that
this dependency upon a specific reflectance model can be relaxed by focusing on
a specific class of objects (e.g., faces), and delegate reflectance estimation
to a deep neural network. A multi-shot strategy based on randomly varying
lighting conditions is eventually discussed. It requires no training or prior
on the reflectance, yet this comes at the price of a dedicated acquisition
setup. Both quantitative and qualitative evaluations illustrate the
effectiveness of the proposed methods on synthetic and real-world scenarios.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence
(T-PAMI), 2019. First three authors contribute equall
A L1-TV Algorithm for Robust Perspective Photometric Stereo with Spatially-Varying Lightings
International audienceWe tackle the problem of perspective 3D-reconstruction of Lambertian surfaces through photometric stereo, in the presence of outliers to Lambert’s law, depth discontinuities, and unknown spatially-varying lightings. To this purpose, we introduce a robust L1-TV variational formulation of the recovery problem where the shape itself is the main unknown, which naturally enforces integrability and permits to avoid integrating the normal field
Variational Uncalibrated Photometric Stereo under General Lighting
Photometric stereo (PS) techniques nowadays remain constrained to an ideal
laboratory setup where modeling and calibration of lighting is amenable. To
eliminate such restrictions, we propose an efficient principled variational
approach to uncalibrated PS under general illumination. To this end, the
Lambertian reflectance model is approximated through a spherical harmonic
expansion, which preserves the spatial invariance of the lighting. The joint
recovery of shape, reflectance and illumination is then formulated as a single
variational problem. There the shape estimation is carried out directly in
terms of the underlying perspective depth map, thus implicitly ensuring
integrability and bypassing the need for a subsequent normal integration. To
tackle the resulting nonconvex problem numerically, we undertake a two-phase
procedure to initialize a balloon-like perspective depth map, followed by a
"lagged" block coordinate descent scheme. The experiments validate efficiency
and robustness of this approach. Across a variety of evaluations, we are able
to reduce the mean angular error consistently by a factor of 2-3 compared to
the state-of-the-art.Comment: Haefner and Ye contributed equall
Transformation d'un dispositif multimédia webcam-écran en un scanner 3D
National audienceNous étudions un dispositif de scannage 3D constitué d’un couple webcam-écran, où l’écran est utilisé comme source lumineuse. Ceci permet de transformer en scanner 3D n’importe quel dispositif multimédia comprenant ces deux éléments (ordinateurs portables, smartphones, tablettes etc.). Un protocole d’étalonnage simplifié est introduit, pour lequel nous démontrons que deux prises de vue sont suffisantes. Une fois cet étalonnage géométrique effectué, nous montrons que le dispositif étudié permet d’effectuer la reconstruction 3D sans ambiguïté, grâce à la technique de stéréophotométrie
- …