Single-shot layered reflectance separation using a polarized light field camera

We present a novel computational photography technique for single shot separation of diffuse/specular reflectance as well as novel angular domain separation of layered reflectance. Our solution consists of a two-way polarized light field (TPLF) camera which simultaneously captures two orthogonal states of polarization. A single photograph of a subject acquired with the TPLF camera under polarized illumination then enables standard separation of diffuse (depolarizing) and polarization preserving specular reflectance using light field sampling. We further demonstrate that the acquired data also enables novel angular separation of layered reflectance including separation of specular reflectance and single scattering in the polarization preserving component, and separation of shallow scattering from deep scattering in the depolarizing component. We apply our approach for efficient acquisition of facial reflectance including diffuse and specular normal maps, and novel separation of photometric normals into layered reflectance normals for layered facial renderings. We demonstrate our proposed single shot layered reflectance separation to be comparable to an existing multi-shot technique that relies on structured lighting while achieving separation results under a variety of illumination conditions

Hyperspectral Modeling of Material Appearance: General Framework, Challenges and Prospects

The main purpose of this tutorial is to address theoretical and practical issues involved in the development of predictive material appearancemodels for interdisciplinary applications within and outside the visible spectral domain. We examine the specific constraints and pitfalls found in each of the key stages of the model development framework, namely data collection, design and evaluation, and discuss alternatives to enhance the effectiveness of the entire process. Although predictive material appearance models developed by computer graphics researchers are usually aimed at realistic image synthesis applications, they also provide valuable support for a myriad of advanced investigations in related areas, such as computer vision, image processing and pattern recognition, which rely on the accurate analysis and interpretation of material appearance attributes in the hyperspectral domain. In fact, their scope of contributions goes beyond the realm of traditional computer science applications. For example, predictive light transport simulations, which are essential for the development of these models, are also regularly beingused by physical and life science researchers to understand andpredict material appearance changes prompted by mechanisms which cannot be fully studied using standard ``wet'' experimental procedures.For completeness, this tutorial also provides an overview of such synergistic research efforts and in silico investigations, which are illustrated by case studies involving the use of hyperspectral material appearance models

Practical Measurement and Reconstruction of Spectral Skin Reflectance

We present two practical methods for measurement of spectral skin reflectance suited for live subjects, and drive a spectral BSSRDF model with appropriate complexity to match skin appearance in photographs, including human faces. Our primary measurement method employs illuminating a subject with two complementary uniform spectral illumination conditions using a multispectral LED sphere to estimate spatially varying parameters of chromophore concentrations including melanin and hemoglobin concentration, melanin blend-type fraction, and epidermal hemoglobin fraction. We demonstrate that our proposed complementary measurements enable higher-quality estimate of chromophores than those obtained using standard broadband illumination, while being suitable for integration with multiview facial capture using regular color cameras. Besides novel optimal measurements under controlled illumination, we also demonstrate how to adapt practical skin patch measurements using a hand-held dermatological skin measurement device, a Miravex Antera 3D camera, for skin appearance reconstruction and rendering. Furthermore, we introduce a novel approach for parameter estimation given the measurements using neural networks which is significantly faster than a lookup table search and avoids parameter quantization. We demonstrate high quality matches of skin appearance with photographs for a variety of skin types with our proposed practical measurement procedures, including photorealistic spectral reproduction and renderings of facial appearance

A Biophysically-Based Model of the Optical Properties of Skin Aging

This paper presents a time-varying, multi-layered biophysically-based model of the optical properties of human skin, suitable for simulating appearance changes due to aging. We have identified the key aspects that cause such changes, both in terms of the structure of skin and its chromophore concentrations, and rely on the extensive medical and optical tissue literature for accurate data. Our model can be expressed in terms of biophysical parameters, optical parameters commonly used in graphics and rendering (such as spectral absorption and scattering coefficients), or more intuitively with higher-level parameters such as age, gender, skin care or skin type. It can be used with any rendering algorithm that uses diffusion profiles, and it allows to automatically simulate different types of skin at different stages of aging, avoiding the need for artistic input or costly capture processes

An Introduction to Light Interaction with Human Skin

Despite the notable progress in physically-based rendering, there is still a long way to go before one can automatically generate predictable images of organic materials such as human skin. In this tutorial, the main physical and biological aspects involved in the processes of propagation and absorption of light by skin tissues are examined. These processes affect not only skin appearance, but also its health. For this reason, they have also been the object of study in biomedical research. The models of light interaction with human skin developed by the biomedical community are mainly aimed at the simulation of skin spectral properties which are used to determine the concentration and distribution of various substances. In computer graphics, the focus has been on the simulation of light scattering properties that affect skin appearance. Computer models used to simulate these spectral and scattering properties are described in this tutorial, and their strengths and limitations discussed. Keywords: natural phenomena, biologically and physically-based rendering

Enhancing Mesh Deformation Realism: Dynamic Mesostructure Detailing and Procedural Microstructure Synthesis

Propomos uma solução para gerar dados de mapas de relevo dinâmicos para simular deformações em superfícies macias, com foco na pele humana. A solução incorpora a simulação de rugas ao nível mesoestrutural e utiliza texturas procedurais para adicionar detalhes de microestrutura estáticos. Oferece flexibilidade além da pele humana, permitindo a geração de padrões que imitam deformações em outros materiais macios, como couro, durante a animação. As soluções existentes para simular rugas e pistas de deformação frequentemente dependem de hardware especializado, que é dispendioso e de difícil acesso. Além disso, depender exclusivamente de dados capturados limita a direção artística e dificulta a adaptação a mudanças. Em contraste, a solução proposta permite a síntese dinâmica de texturas que se adaptam às deformações subjacentes da malha de forma fisicamente plausível. Vários métodos foram explorados para sintetizar rugas diretamente na geometria, mas sofrem de limitações como auto-interseções e maiores requisitos de armazenamento. A intervenção manual de artistas na criação de mapas de rugas e mapas de tensão permite controle, mas pode ser limitada em deformações complexas ou onde maior realismo seja necessário. O nosso trabalho destaca o potencial dos métodos procedimentais para aprimorar a geração de padrões de deformação dinâmica, incluindo rugas, com maior controle criativo e sem depender de dados capturados. A incorporação de padrões procedimentais estáticos melhora o realismo, e a abordagem pode ser estendida além da pele para outros materiais macios.We propose a solution for generating dynamic heightmap data to simulate deformations for soft surfaces, with a focus on human skin. The solution incorporates mesostructure-level wrinkles and utilizes procedural textures to add static microstructure details. It offers flexibility beyond human skin, enabling the generation of patterns mimicking deformations in other soft materials, such as leater, during animation. Existing solutions for simulating wrinkles and deformation cues often rely on specialized hardware, which is costly and not easily accessible. Moreover, relying solely on captured data limits artistic direction and hinders adaptability to changes. In contrast, our proposed solution provides dynamic texture synthesis that adapts to underlying mesh deformations. Various methods have been explored to synthesize wrinkles directly to the geometry, but they suffer from limitations such as self-intersections and increased storage requirements. Manual intervention by artists using wrinkle maps and tension maps provides control but may be limited to the physics-based simulations. Our research presents the potential of procedural methods to enhance the generation of dynamic deformation patterns, including wrinkles, with greater creative control and without reliance on captured data. Incorporating static procedural patterns improves realism, and the approach can be extended to other soft-materials beyond skin

Balancing Fidelity and Performance in Iridal Light Transport Simulations Aimed at Interactive Applications

Specific light transport models based on first-principles approaches have been proposed for complex organic materials such as human skin and blood. The driving force behind these efforts has been the high-fidelity reproduction of material appearance attributes without one having to rely on the manipulation of ad hoc parameters. These models, however, are usually considered excessively time consuming for rendering applications requiring interactive rates. In this thesis, we address this open problem with respect to one of the most challenging of these organic materials, namely the human iris. More specifically, we present a framework that consists in the careful configuration of algorithms employed by a biophysically-based iridal light transport model on the CUDA (Compute Unified Device Architecture) parallel computing platform. We then investigate the sensitivity of iridal appearance attributes to key model running parameters, namely spectral resolution and number of sample rays, in order to obtain a practical balance between appearance fidelity and performance on this platform. The results of our investigation indicate that predictive light transport simulations can be effectively employed in the generation of iridal images that are not only believable, but also controlled by biophysically meaningful parameters. Although our investigation is centered at the human iris, we believe that it can be viewed as a proof of concept, and the proposed configuration strategies and parameter space explorations can be employed to obtain similar results for other organic materials

Validation of a Method to Estimate Skin Spectral Reflectance Using a Digital Camera.

The accurate measurement of skin color and skin spectral reflectance is becoming increasingly desirable due to its application across several domains, including medical, cosmetics, graphic arts, automation, and social science fields. While there exist robust ways to accurately measure color and spectral reflectance, these methods typically require the use of specialized instruments which are often expensive, invasive, and require expert training. Therefore, it would clearly be advantageous to develop methods that can extract accurate colorimetric and spectral data from readily-available, inexpensive digital RGB cameras. Such methodology involves overcoming several fundamental obstacles due to the limitations of RGB camera data. The current paper reviews the importance of accurate skin color and skin spectral reflectance to several domains. The paper continues by describing an existing methodology (i.e., ColourWorker) that overcomes the limitations inherent in using RGB camera data to estimate spectral reflectance. Finally, the paper presents two experiments that test the validity of ColourWorker in estimating skin spectral reflectance. Experiment 1 compares the ground-truth skin spectral reflectance data obtained from a spectroradiometer (taken from the face of volunteers at rest) to spectral reflectance data estimated from an RGB camera using ColourWorker. Experiment 2 compares the ground-truth skin spectral reflectance data obtained from a spectroradiometer (taken from the hand of volunteers with changing physiological states) to spectral reflectance data estimated from an RGB camera using ColourWorker. The results show good performance in ColourWorker’s ability to estimate skin spectral reflectance, and suggest that performance can be improved with careful consideration of reference spectra