Scalable Multi-Class Gaussian Process Classification via Expectation Propagation

Gaussian processes are non-parametric models that can be used to carry out supervised and unsupervised learning tasks. As they are non-parametric models, their complexity grows with the number of data instances, and as a consequence, they can be used to explain complex phenomena associated with the training dataset. They are also very useful to introduce a priori knowledge in the learning problem, because the characteristics that they can describe are given by a covariance function. Finally, these models are Bayesian models, thus they allow to obtain the uncertainty of the predictions and perform model comparison in an automated way. Despite all these advantages, in practice Gaussian processes have certain limitations. The first one is that the computations needed to train the model are only tractable in regression problems with Gaussian additive noise, and for any other case they need to be approximated. The other problem is their scalability, given that the training cost is cubic with respect to the number of observed data points N. In this master thesis, we propose a method for multiclass classification with Gaussian processes that scales well to very large datasets. For that, it uses the Expectation Propagation algorithm, along with the Fully Independent Training Conditional approximation (which introduces M N pseudo-inputs), stochastic gradients and some extra assumptions that reduce the training cost to O(M3). Experimental results show that this method is competitive with other approaches based on variational inference