Reflectance Adaptive Filtering Improves Intrinsic Image Estimation

Separating an image into reflectance and shading layers poses a challenge for learning approaches because no large corpus of precise and realistic ground truth decompositions exists. The Intrinsic Images in the Wild~(IIW) dataset provides a sparse set of relative human reflectance judgments, which serves as a standard benchmark for intrinsic images. A number of methods use IIW to learn statistical dependencies between the images and their reflectance layer. Although learning plays an important role for high performance, we show that a standard signal processing technique achieves performance on par with current state-of-the-art. We propose a loss function for CNN learning of dense reflectance predictions. Our results show a simple pixel-wise decision, without any context or prior knowledge, is sufficient to provide a strong baseline on IIW. This sets a competitive baseline which only two other approaches surpass. We then develop a joint bilateral filtering method that implements strong prior knowledge about reflectance constancy. This filtering operation can be applied to any intrinsic image algorithm and we improve several previous results achieving a new state-of-the-art on IIW. Our findings suggest that the effect of learning-based approaches may have been over-estimated so far. Explicit prior knowledge is still at least as important to obtain high performance in intrinsic image decompositions.Comment: CVPR 201

What image features guide lightness perception?

Lightness constancy is the ability to perceive black and white surface colors under a wide range of lighting conditions. This fundamental visual ability is not well understood, and current theories differ greatly on what image features are important for lightness perception. Here we measured classification images for human observers and four models of lightness perception to determine which image regions influenced lightness judgments. The models were a high-pass-filter model, an oriented difference-of-Gaussians model, an anchoring model, and an atmospheric-link-function model. Human and model observers viewed three variants of the argyle illusion (Adelson, 1993) and judged which of two test patches appeared lighter. Classification images showed that human lightness judgments were based on local, anisotropic stimulus regions that were bounded by regions of uniform lighting. The atmospheric-link-function and anchoring models predicted the lightness illusion perceived by human observers, but the high-pass-filter and oriented-difference-of-Gaussians models did not. Furthermore, all four models produced classification images that were qualitatively different from those of human observers, meaning that the model lightness judgments were guided by different image regions than human lightness judgments. These experiments provide a new test of models of lightness perception, and show that human observers' lightness computations can be highly local, as in low-level models, and nevertheless depend strongly on lighting boundaries, as suggested by midlevel models.York University Librarie

Perception of Lighting and Reflectance in Real and Synthetic Stimuli

The human visual system estimates the proportion of light reflected off of a surface despite variable lighting in a scene, a phenomenon known as lightness constancy. Classically, lightness constancy has been explained as a 'discounting' of the lighting intensity (Helmholtz, 1866), and this continues to be a common view today (e.g., Brainard & Maloney, 2011). However, Logvinenko and Maloney (2006) have made a radically different claim that the human visual system does not have any perceptual access to an estimation of lightness. The experiments described in Chapter 2 use a novel experimental paradigm to test this new theory proposed by Logvinenko and Maloney. We provide evidence against Logvinenko and Maloney's theory of lightness perception while adding to existing evidence that the visual system has good lightness constancy. In Chapter 3, we manipulate screen colour and texture cues to test the realism of computer-generated stimuli. We find that by matching the chromaticity of an LCD screen to the surrounding lighting and using a realistic texture, LCD screens can be made to appear similar to physical paper. Finally, Chapter 4 is an extension of the ideas from Chapter 3, in which the knowledge about how to adjust color and texture cues on an LCD monitor is applied to a lightness matching task. Here, the LCD screen is a small part of a larger physical setup. Additionally, levels of lightness constancy are compared across physical and simulated surfaces in the same novel experimental paradigm in Chapters 2 and 4. We find that physical and simulated surfaced elicit different levels of lightness constancy on the same task

What is the relationship between lightness and perceived illumination

Surface reflectance and illumination level, which are confounded in the retinal image, must be disentangled by the visual system and a theory of lightness must explain how. Thus, a theory of surface lightness should also be a theory of perceived illumination and describe the relationship between them. Perceived illumination and perceived grey values have been measured using a new technique. Looking into a vision tunnel, observers saw two square apertures in the far wall, each revealing a patch of wall composed of two shades of grey. They adjusted the illumination level in one aperture to match that in the other. The stimuli placed in the apertures varied in luminance range, spatial frequency, and relative area. Results show that 1) illumination is matched for highest luminance (with no effect of spatial frequency). Combined with earlier findings that lightness is anchored by highest luminance, this supports Koffka’s suggestion that lightness and perceived illumination are coupled in an invariant way. 2) Changes in the relative area of the light and dark shades produced complementary influences on perceived illumination and surface lightness. That is, when stimulus conditions evoke a conflict between anchoring the highest luminance at white and anchoring the largest area at white, enlarging the darker shade causes its lightness to increase and the perceived illumination to decrease by the same amount, 3 further supporting Koffka. 3) These findings allow perceived illumination level to now be systematically incorporated into anchoring theory, which until this point has been solely a theory of surface lightness

Evaluation and improvement of the workflow of digital imaging of fine art reproduction in museums

Fine arts refer to a broad spectrum of art formats, ie~painting, calligraphy, photography, architecture, and so forth. Fine art reproductions are to create surrogates of the original artwork that are able to faithfully deliver the aesthetics and feelings of the original. Traditionally, reproductions of fine art are made in the form of catalogs, postcards or books by museums, libraries, archives, and so on (hereafter called museums for simplicity). With the widespread adoption of digital archiving in museums, more and more artwork is reproduced to be viewed on a display. For example, artwork collections are made available through museum websites and Google Art Project for art lovers to view on their own displays. In the thesis, we study the fine art reproduction of paintings in the form of soft copy viewed on displays by answering four questions: (1) what is the impact of the viewing condition and original on image quality evaluation? (2) can image quality be improved by avoiding visual editing in current workflows of fine art reproduction? (3) can lightweight spectral imaging be used for fine art reproduction? and (4) what is the performance of spectral reproductions compared with reproductions by current workflows? We started with evaluating the perceived image quality of fine art reproduction created by representative museums in the United States under controlled and uncontrolled environments with and without the presence of the original artwork. The experimental results suggest that the image quality is highly correlated with the color accuracy of the reproduction only when the original is present and the reproduction is evaluated on a characterized display. We then examined the workflows to create these reproductions, and found that current workflows rely heavily on visual editing and retouching (global and local color adjustments on the digital reproduction) to improve the color accuracy of the reproduction. Visual editing and retouching can be both time-consuming and subjective in nature (depending on experts\u27 own experience and understanding of the artwork) lowering the efficiency of artwork digitization considerably. We therefore propose to improve the workflow of fine art reproduction by (1) automating the process of visual editing and retouching in current workflows based on RGB acquisition systems and by (2) recovering the spectral reflectance of the painting with off-the-shelf equipment under commonly available lighting conditions. Finally, we studied the perceived image quality of reproductions created by current three-channel (RGB) workflows with those by spectral imaging and those based on an exemplar-based method

Band-Sifting Decomposition for Image-Based Material Editing

Photographers often "prep" their subjects to achieve various effects; for example, toning down overly shiny skin, covering blotches, etc. Making such adjustments digitally after a shoot is possible, but difficult without good tools and good skills. Making such adjustments to video footage is harder still. We describe and study a set of 2D image operations, based on multiscale image analysis, that are easy and straightforward and that can consistently modify perceived material properties. These operators first build a subband decomposition of the image and then selectively modify the coefficients within the subbands. We call this selection process band sifting. We show that different siftings of the coefficients can be used to modify the appearance of properties such as gloss, smoothness, pigmentation, or weathering. The band-sifting operators have particularly striking effects when applied to faces; they can provide "knobs" to make a face look wetter or drier, younger or older, and with heavy or light variation in pigmentation. Through user studies, we identify a set of operators that yield consistent subjective effects for a variety of materials and scenes. We demonstrate that these operators are also useful for processing video sequences

Neuro-Architecture

Architectural design and neuroscience at first glance may appear to be two seemingly different fields but for centuries intuitively, architects have been designing based on the principles of neuroscience. Architects through trial and error have gained knowledge of specific architectural elements and the potential these elements have to affect the user. Recently this intuition has been coined “neuro-architecture”. With the advancement of technology neuroscientist can accurately conclude how the human body will react to specific architectural stimuli. The proposal is focused on encouraging and furthering the symbiotic relationship between architecture and neuroscience in an attempt to promote architectural design that moves and elevates the human condition. The purpose of this thesis is to investigate the findings of neuroscience and promote their implementation into architectural design, creating a deeper understanding of how the human body relates to architectural surroundings. The methodology assumed closely follows the research typologies used in evidence-based design. The first is a literature review of the findings in neuroscience research and their application to architectural design. Second is an understanding of the anatomy of the body, the senses, and neurobiology as this is the basis in determining the body’s primal reaction to architectural stimuli. The final step of the process will be to create a prototypical design in which research findings bridged and reinforce the connection between neuroscience and architecture, resulting in a design that potentially has the ability to elevate the human experience