Illumination Normalization of Face Images with Cast Shadows

Abstract We propose a method for extractin

Class-Based Feature Matching Across Unrestricted Transformations

We develop a novel method for class-based feature matching across large changes in viewing conditions. The method is based on the property that when objects share a similar part, the similarity is preserved across viewing conditions. Given a feature and a training set of object images, we first identify the subset of objects that share this feature. The transformation of the feature's appearance across viewing conditions is determined mainly by properties of the feature, rather than of the object in which it is embedded. Therefore, the transformed feature will be shared by approximately the same set of objects. Based on this consistency requirement, corresponding features can be reliably identified from a set of candidate matches. Unlike previous approaches, the proposed scheme compares feature appearances only in similar viewing conditions, rather than across different viewing conditions. As a result, the scheme is not restricted to locally planar objects or affine transformations. The approach also does not require examples of correct matches. We show that by using the proposed method, a dense set of accurate correspondences can be obtained. Experimental comparisons demonstrate that matching accuracy is significantly improved over previous schemes. Finally, we show that the scheme can be successfully used for invariant object recognition

CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering

Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire. We explore the use of synthetic data for training CNN-based intrinsic image decomposition models, then applying these learned models to real-world images. To that end, we present \ICG, a new, large-scale dataset of physically-based rendered images of scenes with full ground truth decompositions. The rendering process we use is carefully designed to yield high-quality, realistic images, which we find to be crucial for this problem domain. We also propose a new end-to-end training method that learns better decompositions by leveraging \ICG, and optionally IIW and SAW, two recent datasets of sparse annotations on real-world images. Surprisingly, we find that a decomposition network trained solely on our synthetic data outperforms the state-of-the-art on both IIW and SAW, and performance improves even further when IIW and SAW data is added during training. Our work demonstrates the suprising effectiveness of carefully-rendered synthetic data for the intrinsic images task.Comment: Paper for 'CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering' published in ECCV, 201

Color image-based shape reconstruction of multi-color objects under general illumination conditions

Humans have the ability to infer the surface reflectance properties and three-dimensional shape of objects from two-dimensional photographs under simple and complex illumination fields. Unfortunately, the reported algorithms in the area of shape reconstruction require a number of simplifying assumptions that result in poor performance in uncontrolled imaging environments. Of all these simplifications, the assumptions of non-constant surface reflectance, globally consistent illumination, and multiple surface views are the most likely to be contradicted in typical environments. In this dissertation, three automatic algorithms for the recovery of surface shape given non-constant reflectance using a single-color image acquired are presented. In addition, a novel method for the identification and removal of shadows from simple scenes is discussed.In existing shape reconstruction algorithms for surfaces of constant reflectance, constraints based on the assumed smoothness of the objects are not explicitly used. Through Explicit incorporation of surface smoothness properties, the algorithms presented in this work are able to overcome the limitations of the previously reported algorithms and accurately estimate shape in the presence of varying reflectance. The three techniques developed for recovering the shape of multi-color surfaces differ in the method through which they exploit the surface smoothness property. They are summarized below:• Surface Recovery using Pre-Segmentation - this algorithm pre-segments the image into distinct color regions and employs smoothness constraints at the color-change boundaries to constrain and recover surface shape. This technique is computationally efficient and works well for images with distinct color regions, but does not perform well in the presence of high-frequency color textures that are difficult to segment.iv• Surface Recovery via Normal Propagation - this approach utilizes local gradient information to propagate a smooth surface solution from points of known orientation. While solution propagation eliminates the need for color-based image segmentation, the quality of the recovered surface can be degraded by high degrees of image noise due to reliance on local information.• Surface Recovery by Global Variational Optimization - this algorithm utilizes a normal gradient smoothness constraint in a non-linear optimization strategy, to iteratively solve for the globally optimal object surface. Because of its global nature, this approach is much less sensitive to noise than the normal propagation is, but requires significantly more computational resources.Results acquired through application of the above algorithms to various synthetic and real image data sets are presented for qualitative evaluation. A quantitative analysis of the algorithms is also discussed for quadratic shapes. The robustness of the three approaches to factors such as segmentation error and random image noise is also explored

Self-Supervised Intrinsic Image Decomposition

Intrinsic decomposition from a single image is a highly challenging task, due to its inherent ambiguity and the scarcity of training data. In contrast to traditional fully supervised learning approaches, in this paper we propose learning intrinsic image decomposition by explaining the input image. Our model, the Rendered Intrinsics Network (RIN), joins together an image decomposition pipeline, which predicts reflectance, shape, and lighting conditions given a single image, with a recombination function, a learned shading model used to recompose the original input based off of intrinsic image predictions. Our network can then use unsupervised reconstruction error as an additional signal to improve its intermediate representations. This allows large-scale unlabeled data to be useful during training, and also enables transferring learned knowledge to images of unseen object categories, lighting conditions, and shapes. Extensive experiments demonstrate that our method performs well on both intrinsic image decomposition and knowledge transfer.Comment: NIPS 2017 camera-ready version, project page: http://rin.csail.mit.edu

Robust face recognition by combining projection-based image correction and decomposed eigenface

This work presents a robust face recognition method, which can work even when an insufficient number of images are registered for each person. The method is composed of image correction and image decomposition, both of which are specified in the normalized image space (NIS). The image correction [(F. Sakaue and T. Shakunaga, 2004), (T. Shakunaga and F. Sakaue, 2002)] is realized by iterative projections of an image to an eigenspace in NIS. It works well for natural images having various kinds of noise, including shadows, reflections, and occlusions. We have proposed decomposition of an eigenface into two orthogonal eigenspaces [T. Shakunaga and K. Shigenari, 2001], and have shown that the decomposition is effective for realizing robust face recognition under various lighting conditions. This work shows that the decomposed eigenface method can be refined by projection-based image correction