Multi-view Learning as a Nonparametric Nonlinear Inter-Battery Factor Analysis

Factor analysis aims to determine latent factors, or traits, which summarize a given data set. Inter-battery factor analysis extends this notion to multiple views of the data. In this paper we show how a nonlinear, nonparametric version of these models can be recovered through the Gaussian process latent variable model. This gives us a flexible formalism for multi-view learning where the latent variables can be used both for exploratory purposes and for learning representations that enable efficient inference for ambiguous estimation tasks. Learning is performed in a Bayesian manner through the formulation of a variational compression scheme which gives a rigorous lower bound on the log likelihood. Our Bayesian framework provides strong regularization during training, allowing the structure of the latent space to be determined efficiently and automatically. We demonstrate this by producing the first (to our knowledge) published results of learning from dozens of views, even when data is scarce. We further show experimental results on several different types of multi-view data sets and for different kinds of tasks, including exploratory data analysis, generation, ambiguity modelling through latent priors and classification.Comment: 49 pages including appendi

Improving Facial Analysis and Performance Driven Animation through Disentangling Identity and Expression

We present techniques for improving performance driven facial animation, emotion recognition, and facial key-point or landmark prediction using learned identity invariant representations. Established approaches to these problems can work well if sufficient examples and labels for a particular identity are available and factors of variation are highly controlled. However, labeled examples of facial expressions, emotions and key-points for new individuals are difficult and costly to obtain. In this paper we improve the ability of techniques to generalize to new and unseen individuals by explicitly modeling previously seen variations related to identity and expression. We use a weakly-supervised approach in which identity labels are used to learn the different factors of variation linked to identity separately from factors related to expression. We show how probabilistic modeling of these sources of variation allows one to learn identity-invariant representations for expressions which can then be used to identity-normalize various procedures for facial expression analysis and animation control. We also show how to extend the widely used techniques of active appearance models and constrained local models through replacing the underlying point distribution models which are typically constructed using principal component analysis with identity-expression factorized representations. We present a wide variety of experiments in which we consistently improve performance on emotion recognition, markerless performance-driven facial animation and facial key-point tracking.Comment: to appear in Image and Vision Computing Journal (IMAVIS

Robust Principal Component Analysis on Graphs

Principal Component Analysis (PCA) is the most widely used tool for linear dimensionality reduction and clustering. Still it is highly sensitive to outliers and does not scale well with respect to the number of data samples. Robust PCA solves the first issue with a sparse penalty term. The second issue can be handled with the matrix factorization model, which is however non-convex. Besides, PCA based clustering can also be enhanced by using a graph of data similarity. In this article, we introduce a new model called "Robust PCA on Graphs" which incorporates spectral graph regularization into the Robust PCA framework. Our proposed model benefits from 1) the robustness of principal components to occlusions and missing values, 2) enhanced low-rank recovery, 3) improved clustering property due to the graph smoothness assumption on the low-rank matrix, and 4) convexity of the resulting optimization problem. Extensive experiments on 8 benchmark, 3 video and 2 artificial datasets with corruptions clearly reveal that our model outperforms 10 other state-of-the-art models in its clustering and low-rank recovery tasks

Average Number of Coherent Modes for Pulse Random Fields

Some consequences of spatio-temporal symmetry for the deterministic decomposition of complex light fields into factorized components are considered. This enables to reveal interrelations between spatial and temporal coherence properties of wave. An estimation of average number of the decomposition terms is obtained in the case of statistical ensemble of light pulses.Comment: LaTeX, 10 pages, no figures; to be published in Proc. SPIE; E-mail: [email protected]

Hemispherical confocal imaging using turtleback reflector

We propose a new imaging method called hemispherical confocal imaging to clearly visualize a particular depth in a 3-D scene. The key optical component is a turtleback reflector which is a specially designed polyhedral mirror. By combining the turtleback reflector with a coaxial pair of a camera and a projector, many virtual cameras and projectors are produced on a hemisphere with uniform density to synthesize a hemispherical aperture. In such an optical device, high frequency illumination can be focused at a particular depth in the scene to visualize only the depth with descattering. Then, the observed views are factorized into masking, attenuation, and texture terms to enhance visualization when obstacles are present. Experiments using a prototype system show that only the particular depth is effectively illuminated and hazes by scattering and attenuation can be recovered even when obstacles exist.Microsoft ResearchJapan Society for the Promotion of Science (Grants-in-Aid For Scientific Research 21680017)Japan Society for the Promotion of Science (Grants-in-Aid For Scientific Research 21650038