146 research outputs found
A Novel Framework for Highlight Reflectance Transformation Imaging
We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091DSURF project (PRIN 2015) funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa
DeepShadow: Neural Shape from Shadow
This paper presents DeepShadow, a one-shot method for recovering the depth
map and surface normals from photometric stereo shadow maps. Previous works
that try to recover the surface normals from photometric stereo images treat
cast shadows as a disturbance. We show that the self and cast shadows not only
do not disturb 3D reconstruction, but can be used alone, as a strong learning
signal, to recover the depth map and surface normals. We demonstrate that 3D
reconstruction from shadows can even outperform shape-from-shading in certain
cases. To the best of our knowledge, our method is the first to reconstruct 3D
shape-from-shadows using neural networks. The method does not require any
pre-training or expensive labeled data, and is optimized during inference time
Surface analysis and visualization from multi-light image collections
Multi-Light Image Collections (MLICs) are stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination that provides large amounts of visual and geometric information. Over the last decades, a wide variety of methods have been devised to extract information from MLICs and have shown its use in different application domains to support daily activities. In this thesis, we present methods that leverage a MLICs for surface analysis and visualization. First, we provide background information: acquisition setup, light calibration and application areas where MLICs have been successfully used for the research of daily analysis work. Following, we discuss the use of MLIC for surface visualization and analysis and available tools used to support the analysis. Here, we discuss methods that strive to support the direct exploration of the captured MLIC, methods that generate relightable models from MLIC, non-photorealistic visualization methods that rely on MLIC, methods that estimate normal map from MLIC and we point out visualization tools used to do MLIC analysis. In chapter 3 we propose novel benchmark datasets (RealRTI, SynthRTI and SynthPS) that can be used to evaluate algorithms that rely on MLIC and discusses available benchmark for validation of photometric algorithms that can be also used to validate other MLIC-based algorithms. In chapter 4, we evaluate the performance of different photometric stereo algorithms using SynthPS for cultural heritage applications. RealRTI and SynthRTI have been used to evaluate the performance of (Neural)RTI method. Then, in chapter 5, we present a neural network-based RTI method, aka NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. In this method using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, particularly in the case of challenging glossy materials. Finally, in chapter 6, we present a method for the detection of crack on the surface of paintings from multi-light image acquisitions and that can be used as well on single images and conclude our presentation
Lightweight Photometric Stereo for Facial Details Recovery
Recently, 3D face reconstruction from a single image has achieved great
success with the help of deep learning and shape prior knowledge, but they
often fail to produce accurate geometry details. On the other hand, photometric
stereo methods can recover reliable geometry details, but require dense inputs
and need to solve a complex optimization problem. In this paper, we present a
lightweight strategy that only requires sparse inputs or even a single image to
recover high-fidelity face shapes with images captured under near-field lights.
To this end, we construct a dataset containing 84 different subjects with 29
expressions under 3 different lights. Data augmentation is applied to enrich
the data in terms of diversity in identity, lighting, expression, etc. With
this constructed dataset, we propose a novel neural network specially designed
for photometric stereo based 3D face reconstruction. Extensive experiments and
comparisons demonstrate that our method can generate high-quality
reconstruction results with one to three facial images captured under
near-field lights. Our full framework is available at
https://github.com/Juyong/FacePSNet.Comment: Accepted to CVPR2020. The source code is available
https://github.com/Juyong/FacePSNe
Photometric stereo with applications in material classification
Ph.DDOCTOR OF PHILOSOPH
A Practical Reflectance Transformation Imaging Pipeline for Surface Characterization in Cultural Heritage
We present a practical acquisition and processing pipeline to characterize the surface structure of cultural heritage objects. Using a free-form Reflectance Transformation Imaging (RTI) approach, we acquire multiple digital photographs of the studied object shot from a stationary camera. In each photograph, a light is freely positioned around the object in order to cover a wide variety of illumination directions. Multiple reflective spheres and white Lambertian surfaces are added to the scene to automatically recover light positions and to compensate for non-uniform illumination. An estimation of geometry and reflectance parameters (e.g., albedo, normals, polynomial texture maps coefficients) is then performed to locally characterize surface properties. The resulting object description is stable and representative enough of surface features to reliably provide a characterization of measured surfaces. We validate our approach by comparing RTI-acquired data with data acquired with a high-resolution microprofilometer.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 66509
- …