Multi-class Gaussian Process Classification with Noisy Inputs

It is a common practice in the machine learning community to assume that the observed data are noise-free in the input attributes. Nevertheless, scenarios with input noise are common in real problems, as measurements are never perfectly accurate. If this input noise is not taken into account, a supervised machine learning method is expected to perform sub-optimally. In this paper, we focus on multi-class classification problems and use Gaussian processes (GPs) as the underlying classifier. Motivated by a data set coming from the astrophysics domain, we hypothesize that the observed data may contain noise in the inputs. Therefore, we devise several multi-class GP classifiers that can account for input noise. Such classifiers can be efficiently trained using variational inference to approximate the posterior distribution of the latent variables of the model. Moreover, in some situations, the amount of noise can be known before-hand. If this is the case, it can be readily introduced in the proposed methods. This prior information is expected to lead to better performance results. We have evaluated the proposed methods by carrying out several experiments, involving synthetic and real data. These include several data sets from the UCI repository, the MNIST data set and a data set coming from astrophysics. The results obtained show that, although the classification error is similar across methods, the predictive distribution of the proposed methods is better, in terms of the test log-likelihood, than the predictive distribution of a classifier based on GPs that ignores input noiseWe would like to thank M. A. Sanchez-Conde, J. Coronado and V. Gammaldi for pointing our attention to the data set that motivated this work, as well as for the discussions concerning the data extraction. We thank as well E. Fernandez-Martınez, A. Suarez and C. M. Alaız-Gudin for useful discussions and feedback about the work. BZ especially acknowledges the hospitality of the Machine Learning group of UAM during the development of this project. BZ is supported by the Programa Atraccion de Talento de la Comunidad de Madrid under grant n. 2017-T2/TIC-5455, from the Spanish MINECO’s “Centro de Excelencia Severo Ochoa” Programme via grant SEV-2016-0597, and from the Comunidad de Madrid project SI1-PJI-2019-00294, of which BZ is the P.I. The authors gratefully acknowledge the use of the facilities of Centro de Computacion Cientıfica (CCC) at Universidad Autonoma de Madrid. The authors also acknowledge financial support from Spanish Plan Nacional I+D+i, grants TIN2016-76406-P. Finally, the authors acknowledge financial support from PID2019-106827GB-I00/AEI/10.13039/50110001103