Discriminative Gaussian Process Latent Variable Model for Classification

Supervised learning is difficult with high dimensional input spacesand very small training sets, but accurate classification may bepossible if the data lie on a low-dimensional manifold. GaussianProcess Latent Variable Models can discover low dimensional manifoldsgiven only a small number of examples, but learn a latent spacewithout regard for class labels. Existing methods for discriminativemanifold learning (e.g., LDA, GDA) do constrain the class distributionin the latent space, but are generally deterministic and may notgeneralize well with limited training data. We introduce a method forGaussian Process Classification using latent variable models trainedwith discriminative priors over the latent space, which can learn adiscriminative latent space from a small training set

poLCA: An R Package for Polytomous Variable Latent Class Analysis

poLCA is a software package for the estimation of latent class and latent class regression models for polytomous outcome variables, implemented in the R statistical computing environment. Both models can be called using a single simple command line. The basic latent class model is a finite mixture model in which the component distributions are assumed to be multi-way cross-classification tables with all variables mutually independent. The latent class regression model further enables the researcher to estimate the effects of covariates on predicting latent class membership. poLCA uses expectation-maximization and Newton-Raphson algorithms to find maximum likelihood estimates of the model parameters

Gibbs Max-margin Topic Models with Data Augmentation

Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the non-smoothness of the margin loss. Existing approaches to building max-margin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional mean-field assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new max-margin loss. Namely, we present Gibbs max-margin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multi-task learning. Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple Gibbs sampling algorithms with no restricting assumptions and no need to solve SVM subproblems. Furthermore, each step of the "augment-and-collapse" Gibbs sampling algorithms has an analytical conditional distribution, from which samples can be easily drawn. Experimental results demonstrate significant improvements on time efficiency. The classification performance is also significantly improved over competitors on binary, multi-class and multi-label classification tasks.Comment: 35 page

Estimation of extended mixed models using latent classes and latent processes: the R package lcmm

The R package lcmm provides a series of functions to estimate statistical models based on linear mixed model theory. It includes the estimation of mixed models and latent class mixed models for Gaussian longitudinal outcomes (hlme), curvilinear and ordinal univariate longitudinal outcomes (lcmm) and curvilinear multivariate outcomes (multlcmm), as well as joint latent class mixed models (Jointlcmm) for a (Gaussian or curvilinear) longitudinal outcome and a time-to-event that can be possibly left-truncated right-censored and defined in a competing setting. Maximum likelihood esimators are obtained using a modified Marquardt algorithm with strict convergence criteria based on the parameters and likelihood stability, and on the negativity of the second derivatives. The package also provides various post-fit functions including goodness-of-fit analyses, classification, plots, predicted trajectories, individual dynamic prediction of the event and predictive accuracy assessment. This paper constitutes a companion paper to the package by introducing each family of models, the estimation technique, some implementation details and giving examples through a dataset on cognitive aging

Variational Bayesian multinomial probit regression with Gaussian process priors

It is well known in the statistics literature that augmenting binary and polychotomous response models with Gaussian latent variables enables exact Bayesian analysis via Gibbs sampling from the parameter posterior. By adopting such a data augmentation strategy, dispensing with priors over regression coefficients in favour of Gaussian Process (GP) priors over functions, and employing variational approximations to the full posterior we obtain efficient computational methods for Gaussian Process classification in the multi-class setting. The model augmentation with additional latent variables ensures full a posteriori class coupling whilst retaining the simple a priori independent GP covariance structure from which sparse approximations, such as multi-class Informative Vector Machines (IVM), emerge in a very natural and straightforward manner. This is the first time that a fully Variational Bayesian treatment for multi-class GP classification has been developed without having to resort to additional explicit approximations to the non-Gaussian likelihood term. Empirical comparisons with exact analysis via MCMC and Laplace approximations illustrate the utility of the variational approximation as a computationally economic alternative to full MCMC and it is shown to be more accurate than the Laplace approximation

On The Measurement Of Illegal Wage Discrimination: The Michael Jordan Paradox

Standard wage discrimination models assume that independent observers are able to distinguish a priori which workers are suffering from discrimination. However, this assumption may be inadequate when severe penalties can be imposed on discriminatory employers. Antidiscrimination laws will induce firms to behave in such a way that independent observers (for instance, lawyers, economists) cannot easily detect discriminatory practices. This problem can be solved by estimating the discriminatory wage gap using finite mixture or latent class models because these procedures do not require the a priori classification of workers. In fact, the standard discrimination model can be seen as a particular case of our method when the probabilities of belonging to a group are fixed (to one or zero). We estimate discrimination coefficients for Germany and United Kingdom using the European Community Household Panel (ECHP). We obtain unambiguous higher discrimination in Germany for a wide set of measuresdiscrimination; wages; latent class model; finite mixture models.