Bilinear probabilistic principal component analysis

Probabilistic principal component analysis (PPCA) is a popular linear latent variable model for performing dimension reduction on 1-D data in a probabilistic manner. However, when used on 2-D data such as images, PPCA suffers from the curse of dimensionality due to the subsequently large number of model parameters. To overcome this problem, we propose in this paper a novel probabilistic model on 2-D data called bilinear PPCA (BPPCA). This allows the establishment of a closer tie between BPPCA and its nonprobabilistic counterpart. Moreover, two efficient parameter estimation algorithms for fitting BPPCA are also developed. Experiments on a number of 2-D synthetic and real-world data sets show that BPPCA is more accurate than existing probabilistic and nonprobabilistic dimension reduction methods.published_or_final_versio

Multiple Correspondence Analysis & the Multilogit Bilinear Model

Multiple Correspondence Analysis (MCA) is a dimension reduction method which plays a large role in the analysis of tables with categorical nominal variables such as survey data. Though it is usually motivated and derived using geometric considerations, in fact we prove that it amounts to a single proximal Newtown step of a natural bilinear exponential family model for categorical data the multinomial logit bilinear model. We compare and contrast the behavior of MCA with that of the model on simulations and discuss new insights on the properties of both exploratory multivariate methods and their cognate models. One main conclusion is that we could recommend to approximate the multilogit model parameters using MCA. Indeed, estimating the parameters of the model is not a trivial task whereas MCA has the great advantage of being easily solved by singular value decomposition and scalable to large data

Supervised Dictionary Learning

It is now well established that sparse signal models are well suited to restoration tasks and can effectively be learned from audio, image, and video data. Recent research has been aimed at learning discriminative sparse models instead of purely reconstructive ones. This paper proposes a new step in that direction, with a novel sparse representation for signals belonging to different classes in terms of a shared dictionary and multiple class-decision functions. The linear variant of the proposed model admits a simple probabilistic interpretation, while its most general variant admits an interpretation in terms of kernels. An optimization framework for learning all the components of the proposed model is presented, along with experimental results on standard handwritten digit and texture classification tasks