A coupled statistical model for face shape recovery from brightness images

We focus on the problem of developing a coupled statistical model that can be used to recover facial shape from brightness images of faces. We study three alternative representations for facial shape. These are the surface height function, the surface gradient, and a Fourier basis representation. We jointly capture variations in intensity and the surface shape representations using a coupled statistical model. The model is constructed by performing principal components analysis on sets of parameters describing the contents of the intensity images and the facial shape representations. By fitting the coupled model to intensity data, facial shape is implicitly recovered from the shape parameters. Experiments show that the coupled model is able to generate accurate shape from out-of-training-sample intensity images

Gender classification based on facial surface normals

In this paper, we perform gender classification based on 2.5D facial surface normals (facial needle-maps), and present two novel principal geodesic analysis (PGA) methods, weighted PGA and supervised PGA, to parameterize the facial needle-maps, and compare their performances with PGA for gender classification. Experimental results demonstrate the feasibility of gender classification based on facial needle-maps, and show that incorporating weights or pairwise relationships of labeled data into PGA improves the gender discriminating powers in the leading eigenvectors and the gender classification accuracy

Video Stream Retrieval of Unseen Queries using Semantic Memory

Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem requires temporal evaluation and the unforeseeable scope of potential queries motivates an approach which can accommodate arbitrary search queries. To account for the breadth of possible queries, we adopt a no-example approach to query retrieval, which uses a query's semantic relatedness to pre-trained concept classifiers. To adapt to shifting video content, we propose memory pooling and memory welling methods that favor recent information over long past content. We identify two stream retrieval tasks, instantaneous retrieval at any particular time and continuous retrieval over a prolonged duration, and propose means for evaluating them. Three large scale video datasets are adapted to the challenge of stream retrieval. We report results for our search methods on the new stream retrieval tasks, as well as demonstrate their efficacy in a traditional, non-streaming video task

Making shape from shading work for real-world images

Abstract. Although shape from shading (SfS) has been studied for almost four decades, the performance of most methods applied to realworld images is still unsatisfactory: This is often caused by oversimplified reflectance and projection models as well as by ignoring light attenuation and nonconstant albedo behavior. We address this problem by proposing a novel approach that combines three powerful concepts: (i) By means of a Chan-Vese segmentation step, we partition the image into regions with homogeneous reflectance properties. (ii) This homogeneity is further improved by an adaptive thresholding that singles out unreliable details which cause fluctuating albedos. Using an inpainting method based on edge-enhancing anisotropic diffusion, structures are filled in such that the albedo does no longer suffer from fluctuations. (iii) Finally a sophisticated SfS method is used that features a perspective projection model, considers physical light attenuation and models specular highlights. In our experiments we demonstrate that each of these ingredients improves the reconstruction quality significantly. Their combination within a single method gives favorable perfomance also for images that are taken under real-world conditions where simpler approaches fail.