A Neural Height-Map Approach for the Binocular Photometric Stereo Problem

In this work we propose a novel, highly practical, binocular photometric stereo (PS) framework, which has same acquisition speed as single view PS, however significantly improves the quality of the estimated geometry. As in recent neural multi-view shape estimation frameworks such as NeRF, SIREN and inverse graphics approaches to multi-view photometric stereo (e.g. PS-NeRF) we formulate shape estimation task as learning of a differentiable surface and texture representation by minimising surface normal discrepancy for normals estimated from multiple varying light images for two views as well as discrepancy between rendered surface intensity and observed images. Our method differs from typical multi-view shape estimation approaches in two key ways. First, our surface is represented not as a volume but as a neural heightmap where heights of points on a surface are computed by a deep neural network. Second, instead of predicting an average intensity as PS-NeRF or introducing lambertian material assumptions as Guo et al., we use a learnt BRDF and perform near-field per point intensity rendering. Our method achieves the state-of-the-art performance on the DiLiGenT-MV dataset adapted to binocular stereo setup as well as a new binocular photometric stereo dataset - LUCES-ST.Comment: WACV 202

Virtual restoration and visualization changes through light: A review

This article belongs to the Special Issue Optical Technologies Applied to Cultural Heritage.The virtual modification of the appearance of an object using lighting technologies has become very important in recent years, since the projection of light on an object allows us to alter its appearance in a virtual and reversible way. Considering the limitation of non-contact when analysing a work of art, these optical techniques have been used in fields of restoration of cultural heritage, allowing us to visualize the work as it was conceived by its author, after a process of acquisition and treatment of the image. Furthermore, the technique of altering the appearance of objects through the projection of light has been used in projects with artistic or even educational purposes. This review has treated the main studies of light projection as a technique to alter the appearance of objects, emphasizing the calibration methods used in each study, taking into account the importance of a correct calibration between devices to carry out this technology. In addition, since the described technique consists of projecting light, and one of the applications is related to cultural heritage, those studies that carry out the design and optimization of lighting systems will be described for a correct appreciation of the works of art, without altering its state of conservationThis work has been funded by project number RTI2018-097633-A-I00 of the Ministry of Science and Innovation of Spain, entitled 'Photonic restoration applied to cultural heritage: Application to Dali's painting: Two Figures.

Photo-Realistic Facial Details Synthesis from Single Image

We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation and combines supervised and unsupervised learning for facial detail synthesis. On proxy generation, we conduct emotion prediction to determine a new expression-informed proxy. On detail synthesis, we present a Deep Facial Detail Net (DFDN) based on Conditional Generative Adversarial Net (CGAN) that employs both geometry and appearance loss functions. For geometry, we capture 366 high-quality 3D scans from 122 different subjects under 3 facial expressions. For appearance, we use additional 20K in-the-wild face images and apply image-based rendering to accommodate lighting variations. Comprehensive experiments demonstrate that our framework can produce high-quality 3D faces with realistic details under challenging facial expressions

TOWARDS CASUAL APPEARANCE CAPTURE BY REFLECTANCE SYMMETRY

Ph.DDOCTOR OF PHILOSOPH