225 research outputs found
Multispectral RTI Analysis of Heterogeneous Artworks
We propose a novel multi-spectral reflectance transformation imaging (MS-RTI) framework for the acquisition and direct analysis of the reflectance behavior of heterogeneous artworks. Starting from free-form acquisitions, we compute per-pixel calibrated multi-spectral appearance profiles, which associate a reflectance value to each sampled light direction and frequency. Visualization, relighting, and feature extraction is performed directly on appearance profile data, applying scattered data interpolation based on Radial Basis Functions to estimate per-pixel reflectance from novel lighting directions. We demonstrate how the proposed solution can convey more insights on the object materials and geometric details compared to classical multi-light methods that rely on low-frequency analytical model fitting eventually mixed with a separate handling of high-frequency components, hence requiring constraining priors on material behavior. The flexibility of our approach is illustrated on two heterogeneous case studies, a painting and a dark shiny metallic sculpture, that showcase feature extraction, visualization, and analysis of high-frequency properties of artworks using multi-light, multi-spectral (Visible, UV and IR) acquisitions.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091the DSURF (PRIN 2015) project funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa
DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
Uncalibrated photometric stereo (UPS) is challenging due to the inherent
ambiguity brought by the unknown light. Although the ambiguity is alleviated on
non-Lambertian objects, the problem is still difficult to solve for more
general objects with complex shapes introducing irregular shadows and general
materials with complex reflectance like anisotropic reflectance. To exploit
cues from shadow and reflectance to solve UPS and improve performance on
general materials, we propose DANI-Net, an inverse rendering framework with
differentiable shadow handling and anisotropic reflectance modeling. Unlike
most previous methods that use non-differentiable shadow maps and assume
isotropic material, our network benefits from cues of shadow and anisotropic
reflectance through two differentiable paths. Experiments on multiple
real-world datasets demonstrate our superior and robust performance.Comment: Accepted by CVPR 202
Surface analysis and visualization from multi-light image collections
Multi-Light Image Collections (MLICs) are stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination that provides large amounts of visual and geometric information. Over the last decades, a wide variety of methods have been devised to extract information from MLICs and have shown its use in different application domains to support daily activities. In this thesis, we present methods that leverage a MLICs for surface analysis and visualization. First, we provide background information: acquisition setup, light calibration and application areas where MLICs have been successfully used for the research of daily analysis work. Following, we discuss the use of MLIC for surface visualization and analysis and available tools used to support the analysis. Here, we discuss methods that strive to support the direct exploration of the captured MLIC, methods that generate relightable models from MLIC, non-photorealistic visualization methods that rely on MLIC, methods that estimate normal map from MLIC and we point out visualization tools used to do MLIC analysis. In chapter 3 we propose novel benchmark datasets (RealRTI, SynthRTI and SynthPS) that can be used to evaluate algorithms that rely on MLIC and discusses available benchmark for validation of photometric algorithms that can be also used to validate other MLIC-based algorithms. In chapter 4, we evaluate the performance of different photometric stereo algorithms using SynthPS for cultural heritage applications. RealRTI and SynthRTI have been used to evaluate the performance of (Neural)RTI method. Then, in chapter 5, we present a neural network-based RTI method, aka NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. In this method using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, particularly in the case of challenging glossy materials. Finally, in chapter 6, we present a method for the detection of crack on the surface of paintings from multi-light image acquisitions and that can be used as well on single images and conclude our presentation
Polarisation photometric stereo
© 2017 This paper concerns a novel approach to fuse two-source photometric stereo (PS) data with polarisation information for complete surface normal recovery for smooth or slightly rough surfaces. PS is a well-established method but is limited in application by its need for three or more well-spaced and known illumination sources and Lambertian reflectance. Polarisation methods are less studied but have shown promise for smooth surfaces under highly controlled capture conditions. However, such methods suffer from inherent ambiguities and the depolarising effects of surface roughness. The method presented in this paper goes some way to overcome these limitations by fusing the most reliable information from PS and polarisation. PS is used with only two sources to deduce a constrained mapping of the surface normal at each point onto a 2D plane. Phase information from polarisation is used to deduce a mapping onto a different plane. The paper then shows how the full surface normal can be obtained from the two mappings. The method is tested on a range of real-world images to demonstrate the advantages over standalone applications of PS or polarisation methods
A Novel Framework for Highlight Reflectance Transformation Imaging
We propose a novel pipeline and related software tools for processing the multi-light image collections (MLICs) acquired in different application contexts to obtain shape and appearance information of captured surfaces, as well as to derive compact relightable representations of them. Our pipeline extends the popular Highlight Reflectance Transformation Imaging (H-RTI) framework, which is widely used in the Cultural Heritage domain. We support, in particular, perspective camera modeling, per-pixel interpolated light direction estimation, as well as light normalization correcting vignetting and uneven non-directional illumination. Furthermore, we propose two novel easy-to-use software tools to simplify all processing steps. The tools, in addition to support easy processing and encoding of pixel data, implement a variety of visualizations, as well as multiple reflectance-model-fitting options. Experimental tests on synthetic and real-world MLICs demonstrate the usefulness of the novel algorithmic framework and the potential benefits of the proposed tools for end-user applications.Terms: "European Union (EU)" & "Horizon 2020" / Action: H2020-EU.3.6.3. - Reflective societies - cultural heritage and European identity / Acronym: Scan4Reco / Grant number: 665091DSURF project (PRIN 2015) funded by the Italian Ministry of University and ResearchSardinian Regional Authorities under projects VIGEC and Vis&VideoLa
Deep Reflectance Maps
Undoing the image formation process and therefore decomposing appearance into
its intrinsic properties is a challenging task due to the under-constraint
nature of this inverse problem. While significant progress has been made on
inferring shape, materials and illumination from images only, progress in an
unconstrained setting is still limited. We propose a convolutional neural
architecture to estimate reflectance maps of specular materials in natural
lighting conditions. We achieve this in an end-to-end learning formulation that
directly predicts a reflectance map from the image itself. We show how to
improve estimates by facilitating additional supervision in an indirect scheme
that first predicts surface orientation and afterwards predicts the reflectance
map by a learning-based sparse data interpolation.
In order to analyze performance on this difficult task, we propose a new
challenge of Specular MAterials on SHapes with complex IllumiNation (SMASHINg)
using both synthetic and real images. Furthermore, we show the application of
our method to a range of image-based editing tasks on real images.Comment: project page: http://homes.esat.kuleuven.be/~krematas/DRM
Efficient photometric stereo on glossy surfaces with wide specular lobes.
Chung, Hin Shun.Thesis (M.Phil.)--Chinese University of Hong Kong, 2008.Includes bibliographical references (leaves 40-43).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Lambertian photometric stereo --- p.1Chapter 1.2 --- Non-Lambertian photometric stereo --- p.3Chapter 1.3 --- Large specular lobe problems --- p.4Chapter 2 --- Related Work --- p.9Chapter 2.1 --- Lambertian photometric stereo --- p.9Chapter 2.2 --- Non-Lambertian photometric stereo --- p.9Chapter 2.2.1 --- Analytic models to reconstruct non-Lambertian surface --- p.9Chapter 2.2.2 --- Reference object based --- p.10Chapter 2.2.3 --- Highlight removal before shape reconstruction --- p.11Chapter 2.2.4 --- Polarization based method --- p.12Chapter 2.2.5 --- Specularity fitting method --- p.12Chapter 2.2.6 --- Photometric stereo with shadow --- p.12Chapter 3 --- Our System --- p.13Chapter 3.1 --- Estimation of global parameters --- p.14Chapter 3.1.1 --- Shadow separation --- p.16Chapter 3.1.2 --- Separation edges of shadow and edges of foreground object --- p.16Chapter 3.1.3 --- Normal estimation using shadow boundary --- p.20Chapter 3.1.4 --- Global parameter estimation and refinement --- p.22Chapter 3.2 --- Surface shape and texture reconstruction --- p.24Chapter 3.3 --- Single material results --- p.25Chapter 4 --- Comparison between Our Method and Direct Specularity Fitting Method --- p.29Chapter 4.1 --- Summary of direct specularity fitting method [9] --- p.29Chapter 4.2 --- Comparison results --- p.31Chapter 5 --- Reconstructing Multiple-Material Surfaces --- p.33Chapter 5.1 --- Multiple material results --- p.34Chapter 6 --- Conclusion --- p.38Bibliography --- p.39Chapter A --- Proof of Surface Normal Projecting to Gradient of Cast Shadow Boundary --- p.4
- …