Perception based heterogeneous subsurface scattering for film

Many real world materials exhibit complex subsurface scattering of light. This internal light interaction creates the perception of translucency for the human visual system. Translucent materials and simulation of the subsurface scattering of light has become an expected necessity for generating warmth and realism in computer generated imagery. The light transport within heterogenous materials, such as marble, has proved challenging to model and render. The current material models available to digital artists have been limited to homogeneous subsurface scattering despite a few publications documenting success at simulating heterogeneous light transport. While the publications successfully simulate this complex phenomenon, the material descriptions have been highly specialized and far from intuitive. By combining the measurable properties of heterogeneous translucent materials with the defining properties of translucency, as perceived by the human visual system, a description of heterogeneous translucent materials that is suitable for artist use in a film production pipeline can be achieved. Development of the material description focuses on integration with the film pipeline, ease of use, and reasonable approximation of heterogeneous translucency based on perception. Methods of material manipulation are explored to determine which properties should be modifiable by artists while maintaining the perception of heterogenous translucency

Bidirectional Appearance Distribution Function for Stylized Shading

We define a new shading tool called a Bidirectional Appearance Distribution Function (BADF) tailored to the direct control of stylized appearance. A BADF can be thought of as defining the appearance of a sphere from all possible illumination directions. Our BADF formulation generalizes and improves upon previous stylized shading techniques by enabling the direct control of shading profiles in screen space, exaggerating surface features in a flexible manner, and letting users control stylized appearance from multiple lighting or viewing directions. This allows users to start from a simple shading behavior, and refine from there towards greater stylization. Our GPU implementation works in real-time, which benefits both editing, and rendering in interactive systems. These features make BADFs an efficient tool for many applications in artistic and scientific illustration domains

Fast simulation of translucent objects

The existing literature provides a wide variety of previous work regarding Subsurface Scattering e ects. These include pre-processed algorithms, and real-time techniques. On the next lines some of these techniques will be discussed and analyzed in order to know those that led us to the algorithm that we presen

Image-based Material Editing

Photo editing software allows digital images to be blurred, warped or re-colored at the touch of a button. However, it is not currently possible to change the material appearance of an object except by painstakingly painting over the appropriate pixels. Here we present a set of methods for automatically replacing one material with another, completely different material, starting with only a single high dynamic range image, and an alpha matte specifying the object. Our approach exploits the fact that human vision is surprisingly tolerant of certain (sometimes enormous) physical inaccuracies. Thus, it may be possible to produce a visually compelling illusion of material transformations, without fully reconstructing the lighting or geometry. We employ a range of algorithms depending on the target material. First, an approximate depth map is derived from the image intensities using bilateral filters. The resulting surface normals are then used to map data onto the surface of the object to specify its material appearance. To create transparent or translucent materials, the mapped data are derived from the object\u27s background. To create textured materials, the mapped data are a texture map. The surface normals can also be used to apply arbitrary bidirectional reflectance distribution functions to the surface, allowing us to simulate a wide range of materials. To facilitate the process of material editing, we generate the HDR image with a novel algorithm, that is robust against noise in individual exposures. This ensures that any noise, which would possibly have affected the shape recovery of the objects adversely, will be removed. We also present an algorithm to automatically generate alpha mattes. This algorithm requires as input two images--one where the object is in focus, and one where the background is in focus--and then automatically produces an approximate matte, indicating which pixels belong to the object. The result is then improved by a second algorithm to generate an accurate alpha matte, which can be given as input to our material editing techniques

Depicting shape, materials and lighting: observation, formulation and implementation of artistic principles

The appearance of a scene results from complex interactions between the geometry, materials and lights that compose that scene. While Computer Graphics algorithms are now capable of simulating these interactions, it comes at the cost of tedious 3D modeling of a virtual scene, which only well-trained artists can do. In contrast, photographs allow the instantaneous capture of a scene, but shape, materials and lighting are difficult to manipulate directly in the image. Drawings can also suggest real or imaginary scenes with a few lines but creating convincing illustrations requires significant artistic skills.The goal of my research is to facilitate the creation and manipulation of shape, materials and lighting in drawings and photographs, for laymen and professional artists alike. This document first presents my work on computer-assisted drawing where I proposed algorithms to automate the depiction of materials in line drawings as well as to estimate a 3D model from design sketches. I also worked on user interfaces to assist beginners in learning traditional drawing techniques. Through the development of these projects I have formalized a general methodology to observe how artists work, deduce artistic principles from these observations, and implement these principles as algorithms. In the second part of this document I present my work on relighting multiple photographs of a scene, for which we first need to estimate the materials and lighting that compose that scene. The main novelty of our approach is to combine image analysis and lighting simulation in order to reason about the scene despite the lack of an accurate 3D model

The role of pigmentation in face perception

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2005.Includes bibliographical references.Faces each have distinct pigmentation as well as shape, which suggests that both cues may play a role in the perception of faces. However, there is a common implicit assumption that pigmentation cues are relatively unimportant, and so the role pigmentation plays in face perception has gone largely unexplored. This thesis is a systematic investigation of the role of pigmentation in face recognition, facial sex classification, and facial attractiveness. The present studies present evidence that pigmentation cues are in fact quite important for face perception. For face recognition, pigmentation cues are about as important as shape cues. Male and female faces differ consistently in their pigmentation, with female faces having more luminance contrast between the eyes and lips and the rest of the face than do male faces. This sex difference in pigmentation is used as a cue for judgments of facial sex classification and facial attractiveness. Together, these results implicate an important role for pigmentation, and open new avenues of research in the perception of faces.by Richard Russell.Ph.D

Digital Light

Light symbolises the highest good, it enables all visual art, and today it lies at the heart of billion-dollar industries. The control of light forms the foundation of contemporary vision. Digital Light brings together artists, curators, technologists and media archaeologists to study the historical evolution of digital light-based technologies. Digital Light provides a critical account of the capacities and limitations of contemporary digital light-based technologies and techniques by tracing their genealogies and comparing them with their predecessor media. As digital light remediates multiple historical forms (photography, print, film, video, projection, paint), the collection draws from all of these histories, connecting them to the digital present and placing them in dialogue with one another. Light is at once universal and deeply historical. The invention of mechanical media (including photography and cinematography) allied with changing print technologies (half-tone, lithography) helped structure the emerging electronic media of television and video, which in turn shaped the bitmap processing and raster display of digital visual media. Digital light is, as Stephen Jones points out in his contribution, an oxymoron: light is photons, particulate and discrete, and therefore always digital. But photons are also waveforms, subject to manipulation in myriad ways. From Fourier transforms to chip design, colour management to the translation of vector graphics into arithmetic displays, light is constantly disciplined to human purposes. In the form of fibre optics, light is now the infrastructure of all our media; in urban plazas and handheld devices, screens have become ubiquitous, and also standardised. This collection addresses how this occurred, what it means, and how artists, curators and engineers confront and challenge the constraints of increasingly normalised digital visual media. While various art pieces and other content are considered throughout the collection, the focus is specifically on what such pieces suggest about the intersection of technique and technology. Including accounts by prominent artists and professionals, the collection emphasises the centrality of use and experimentation in the shaping of technological platforms. Indeed, a recurring theme is how techniques of previous media become technologies, inscribed in both digital software and hardware. Contributions include considerations of image-oriented software and file formats; screen technologies; projection and urban screen surfaces; histories of computer graphics, 2D and 3D image editing software, photography and cinematic art; and transformations of light-based art resulting from the distributed architectures of the internet and the logic of the database. Digital Light brings together high profile figures in diverse but increasingly convergent fields, from academy award-winner and co-founder of Pixar, Alvy Ray Smith to feminist philosopher Cathryn Vasseleu

Digital light

Light symbolises the highest good, it enables all visual art, and today it lies at the heart of billion-dollar industries. The control of light forms the foundation of contemporary vision. Digital Light brings together artists, curators, technologists and media archaeologists to study the historical evolution of digital light-based technologies. Digital Light provides a critical account of the capacities and limitations of contemporary digital light-based technologies and techniques by tracing their genealogies and comparing them with their predecessor media. As digital light remediates multiple historical forms (photography, print, film, video, projection, paint), the collection draws from all of these histories, connecting them to the digital present and placing them in dialogue with one another.Light is at once universal and deeply historical. The invention of mechanical media (including photography and cinematography) allied with changing print technologies (half-tone, lithography) helped structure the emerging electronic media of television and video, which in turn shaped the bitmap processing and raster display of digital visual media. Digital light is, as Stephen Jones points out in his contribution, an oxymoron: light is photons, particulate and discrete, and therefore always digital. But photons are also waveforms, subject to manipulation in myriad ways. From Fourier transforms to chip design, colour management to the translation of vector graphics into arithmetic displays, light is constantly disciplined to human purposes. In the form of fibre optics, light is now the infrastructure of all our media; in urban plazas and handheld devices, screens have become ubiquitous, and also standardised. This collection addresses how this occurred, what it means, and how artists, curators and engineers confront and challenge the constraints of increasingly normalised digital visual media.While various art pieces and other content are considered throughout the collection, the focus is specifically on what such pieces suggest about the intersection of technique and technology. Including accounts by prominent artists and professionals, the collection emphasises the centrality of use and experimentation in the shaping of technological platforms. Indeed, a recurring theme is how techniques of previous media become technologies, inscribed in both digital software and hardware. Contributions include considerations of image-oriented software and file formats; screen technologies; projection and urban screen surfaces; histories of computer graphics, 2D and 3D image editing software, photography and cinematic art; and transformations of light-based art resulting from the distributed architectures of the internet and the logic of the database.Digital Light brings together high profile figures in diverse but increasingly convergent fields, from academy award-winner and co-founder of Pixar, Alvy Ray Smith to feminist philosopher Cathryn Vasseleu

Perceiving Translucent Materials

Many materials — such as wax, glass, fruit flesh, and human skin — transmit as well as reflect light. When light passes through an object, it gives the object a characteristic appearance of transparency or translucency. How do we identify materials that transmit light? What are the perceptual dimensions of translucency? What image properties make an object look transmissive rather than opaque? We use a combination of ecological optics and psychophysics to address these questions. Almost all research on the perception of transmissive materials is based on Metelli's linear ‘episcotister’ model. Unfortunately, the episcotister is a poor physical model of light transport in real materials. It ignores: (i) specular reflection, (ii) surface ‘frosting’ (as in ‘frosted’ glass), (iii) refraction and (iv) sub-surface scatter. These shortcomings have profound consequences for the perception of translucency. They affect both the perceptual parameters of transparent materials and the image properties that make objects look transmissive. Recent advances in computer graphics (Jensen et al. SIGGRAPH, 2001) allow us to simulate translucent materials realistically. We can now systematically vary sub-surface light scatter, while holding shape, lighting and viewpoint constant. This allows us to identify image properties that are diagnostic of translucency. We discuss how image statistics based on luminance, contrast, orientation, and scale contribute to the perception of translucency, as well as other cues such as highlights and shadows. Using psychophysical matching tasks, we measure how refractive index and translucency appear to change as the object or lights are moved. Importantly, we find that traditional cues to transparency (e.g. X-junctions and visibility of the underlying layer) are not necessary, and in fact rarely occur in realistic images of transparent objects. Indeed, subjects can estimate an object's translucency even when it is floating in a featureless void