Single-shot layered reflectance separation using a polarized light field camera

We present a novel computational photography technique for single shot separation of diffuse/specular reflectance as well as novel angular domain separation of layered reflectance. Our solution consists of a two-way polarized light field (TPLF) camera which simultaneously captures two orthogonal states of polarization. A single photograph of a subject acquired with the TPLF camera under polarized illumination then enables standard separation of diffuse (depolarizing) and polarization preserving specular reflectance using light field sampling. We further demonstrate that the acquired data also enables novel angular separation of layered reflectance including separation of specular reflectance and single scattering in the polarization preserving component, and separation of shallow scattering from deep scattering in the depolarizing component. We apply our approach for efficient acquisition of facial reflectance including diffuse and specular normal maps, and novel separation of photometric normals into layered reflectance normals for layered facial renderings. We demonstrate our proposed single shot layered reflectance separation to be comparable to an existing multi-shot technique that relies on structured lighting while achieving separation results under a variety of illumination conditions

Practical Measurement and Reconstruction of Spectral Skin Reflectance

We present two practical methods for measurement of spectral skin reflectance suited for live subjects, and drive a spectral BSSRDF model with appropriate complexity to match skin appearance in photographs, including human faces. Our primary measurement method employs illuminating a subject with two complementary uniform spectral illumination conditions using a multispectral LED sphere to estimate spatially varying parameters of chromophore concentrations including melanin and hemoglobin concentration, melanin blend-type fraction, and epidermal hemoglobin fraction. We demonstrate that our proposed complementary measurements enable higher-quality estimate of chromophores than those obtained using standard broadband illumination, while being suitable for integration with multiview facial capture using regular color cameras. Besides novel optimal measurements under controlled illumination, we also demonstrate how to adapt practical skin patch measurements using a hand-held dermatological skin measurement device, a Miravex Antera 3D camera, for skin appearance reconstruction and rendering. Furthermore, we introduce a novel approach for parameter estimation given the measurements using neural networks which is significantly faster than a lookup table search and avoids parameter quantization. We demonstrate high quality matches of skin appearance with photographs for a variety of skin types with our proposed practical measurement procedures, including photorealistic spectral reproduction and renderings of facial appearance

Hyperspectral Modeling of Material Appearance: General Framework, Challenges and Prospects

The main purpose of this tutorial is to address theoretical and practical issues involved in the development of predictive material appearancemodels for interdisciplinary applications within and outside the visible spectral domain. We examine the specific constraints and pitfalls found in each of the key stages of the model development framework, namely data collection, design and evaluation, and discuss alternatives to enhance the effectiveness of the entire process. Although predictive material appearance models developed by computer graphics researchers are usually aimed at realistic image synthesis applications, they also provide valuable support for a myriad of advanced investigations in related areas, such as computer vision, image processing and pattern recognition, which rely on the accurate analysis and interpretation of material appearance attributes in the hyperspectral domain. In fact, their scope of contributions goes beyond the realm of traditional computer science applications. For example, predictive light transport simulations, which are essential for the development of these models, are also regularly beingused by physical and life science researchers to understand andpredict material appearance changes prompted by mechanisms which cannot be fully studied using standard ``wet'' experimental procedures.For completeness, this tutorial also provides an overview of such synergistic research efforts and in silico investigations, which are illustrated by case studies involving the use of hyperspectral material appearance models

A Biophysically-Based Model of the Optical Properties of Skin Aging

This paper presents a time-varying, multi-layered biophysically-based model of the optical properties of human skin, suitable for simulating appearance changes due to aging. We have identified the key aspects that cause such changes, both in terms of the structure of skin and its chromophore concentrations, and rely on the extensive medical and optical tissue literature for accurate data. Our model can be expressed in terms of biophysical parameters, optical parameters commonly used in graphics and rendering (such as spectral absorption and scattering coefficients), or more intuitively with higher-level parameters such as age, gender, skin care or skin type. It can be used with any rendering algorithm that uses diffusion profiles, and it allows to automatically simulate different types of skin at different stages of aging, avoiding the need for artistic input or costly capture processes

DistancePPG: Robust non-contact vital signs monitoring using a camera

Vital signs such as pulse rate and breathing rate are currently measured using contact probes. But, non-contact methods for measuring vital signs are desirable both in hospital settings (e.g. in NICU) and for ubiquitous in-situ health tracking (e.g. on mobile phone and computers with webcams). Recently, camera-based non-contact vital sign monitoring have been shown to be feasible. However, camera-based vital sign monitoring is challenging for people with darker skin tone, under low lighting conditions, and/or during movement of an individual in front of the camera. In this paper, we propose distancePPG, a new camera-based vital sign estimation algorithm which addresses these challenges. DistancePPG proposes a new method of combining skin-color change signals from different tracked regions of the face using a weighted average, where the weights depend on the blood perfusion and incident light intensity in the region, to improve the signal-to-noise ratio (SNR) of camera-based estimate. One of our key contributions is a new automatic method for determining the weights based only on the video recording of the subject. The gains in SNR of camera-based PPG estimated using distancePPG translate into reduction of the error in vital sign estimation, and thus expand the scope of camera-based vital sign monitoring to potentially challenging scenarios. Further, a dataset will be released, comprising of synchronized video recordings of face and pulse oximeter based ground truth recordings from the earlobe for people with different skin tones, under different lighting conditions and for various motion scenarios.Comment: 24 pages, 11 figure

An Introduction to Light Interaction with Human Skin

Despite the notable progress in physically-based rendering, there is still a long way to go before one can automatically generate predictable images of organic materials such as human skin. In this tutorial, the main physical and biological aspects involved in the processes of propagation and absorption of light by skin tissues are examined. These processes affect not only skin appearance, but also its health. For this reason, they have also been the object of study in biomedical research. The models of light interaction with human skin developed by the biomedical community are mainly aimed at the simulation of skin spectral properties which are used to determine the concentration and distribution of various substances. In computer graphics, the focus has been on the simulation of light scattering properties that affect skin appearance. Computer models used to simulate these spectral and scattering properties are described in this tutorial, and their strengths and limitations discussed. Keywords: natural phenomena, biologically and physically-based rendering

Analysis of image noise in multispectral color acquisition

The design of a system for multispectral image capture will be influenced by the imaging application, such as image archiving, vision research, illuminant modification or improved (trichromatic) color reproduction. A key aspect of the system performance is the effect of noise, or error, when acquiring multiple color image records and processing of the data. This research provides an analysis that allows the prediction of the image-noise characteristics of systems for the capture of multispectral images. The effects of both detector noise and image processing quantization on the color information are considered, as is the correlation between the errors in the component signals. The above multivariate error-propagation analysis is then applied to an actual prototype system. Sources of image noise in both digital camera and image processing are related to colorimetric errors. Recommendations for detector characteristics and image processing for future systems are then discussed

Spectral imaging of human portraits and image quality

This dissertation addresses the problem of capturing spectral images for human portraits and evaluating image quality of spectral images. A new spectral imaging approach is proposed in this dissertation for spectral images of human portraits. Thorough statistical analysis is performed for spectral reflectances from various races and different face parts. A spectral imaging system has been designed and calibrated for human portraits. The calibrated imaging system has the ability to represent not only the facial skin but also the spectra of lips, eyes and hair from various races as well. The generated spectral images can be applied to color-imaging system design and analysis. To evaluate the image quality of spectral imaging systems, a visual psychophysical image quality experiment has been performed in this dissertation. The spectral images were simulated based on real spectral imaging system. Meaningful image quality results have been obtained for spectral images generated from different spectral imaging systems. To bridge the gap between the physical measures and subjective visual perceptions of image quality, four image distortion factors were defined. Image quality metrics were obtained and evaluated based statistical analysis and multiple analysis. The image quality metrics have high correlation with subjective assessment for image quality. The image quality contribution of the distortion factors were evaluated. As an extension of the work other researchers in MCSL have initiated, this dissertation research will, working with other researchers in MCSL, put effort to build a publicly accessible database of spectral images, Lippmann2000