Practical Bayesian optimization in the presence of outliers

Inference in the presence of outliers is an important field of research as outliers are ubiquitous and may arise across a variety of problems and domains. Bayesian optimization is method that heavily relies on probabilistic inference. This allows outstanding sample efficiency because the probabilistic machinery provides a memory of the whole optimization process. However, that virtue becomes a disadvantage when the memory is populated with outliers, inducing bias in the estimation. In this paper, we present an empirical evaluation of Bayesian optimization methods in the presence of outliers. The empirical evidence shows that Bayesian optimization with robust regression often produces suboptimal results. We then propose a new algorithm which combines robust regression (a Gaussian process with Student-t likelihood) with outlier diagnostics to classify data points as outliers or inliers. By using an scheduler for the classification of outliers, our method is more efficient and has better convergence over the standard robust regression. Furthermore, we show that even in controlled situations with no expected outliers, our method is able to produce better results.Comment: 10 pages (2 of references), 6 figures, 1 algorith

Laplace approximation and natural gradient for Gaussian process regression with heteroscedastic Student-t model

We propose the Laplace method to derive approximate inference for Gaussian process (GP) regression in the location and scale parameters of the student-t probabilistic model. This allows both mean and variance of data to vary as a function of covariates with the attractive feature that the student-t model has been widely used as a useful tool for robustifying data analysis. The challenge in the approximate inference for the model, lies in the analytical intractability of the posterior distribution and the lack of concavity of the log-likelihood function. We present the natural gradient adaptation for the estimation process which primarily relies on the property that the student-t model naturally has orthogonal parametrization. Due to this particular property of the model the Laplace approximation becomes significantly more robust than the traditional approach using Newton’s methods. We also introduce an alternative Laplace approximation by using model’s Fisher information matrix. According to experiments this alternative approximation provides very similar posterior approximations and predictive performance to the traditional Laplace approximation with model’s Hessian matrix. However, the proposed Laplace–Fisher approximation is faster and more stable to calculate compared to the traditional Laplace approximation. We also compare both of these Laplace approximations with the Markov chain Monte Carlo (MCMC) method. We discuss how our approach can, in general, improve the inference algorithm in cases where the probabilistic model assumed for the data is not log-concave.Peer reviewe

PENGARUH INTENSITAS PERIKLANAN, CITRA PENDIDIKAN POLITEKNIK TERHADAP SIKAP SISWA SMA DI KOTA AMBON UNTUK MENDAFTAR KULIAH PADA PRODI ADMINISTRASI NIAGA POLITEKNIK NEGERI AMBON YANG DIMEDIASI OLEH TINGKAT MOTIVASI SISWA

Education is something that has important priorities now. Education that is good to be the investment capital of the future, education is also beneficial for one's life, good education will determine one's career in the world of work so that it becomes more professional. This study uses the Elaboration Likelihood Model Theory to see how the message delivered by Ambon State Polytechnic through marketing communication and find out how consumers respond to communication messages delivered so that consumers get attention so that they are interested in enrolling in Business Administration Study Programs. The purpose of this study was to determine the intensity of advertising, the image of Polytechnic education on the attitudes of students to enroll in lectures at the Ambon State Polytechnic Business Administration Study Program mediated by the level of student motivation. This research is a quantitative research with explanatory type. The sampling technique in this study was random sampling with a sample of 260 students. After being tested for validity and reality, then Barron-Kenny Path Analysis and Sobel Test were applied to test the significance of the mediation of the data analysis process using linear regression and multiple regression with SPSS assistance (Statistical Package for the Social Sciences). Overall, this study concluded that all variables have a positive effect with a significance value of 0.05. in the hypothesis test it is known that there is a positive relationship between the intensity of the ad with the attitude of the students where t-count is 7.864> t table 1.6524. thus this hypothesis was accepted in this study. then the image of Polytechnic education on student attitudes has a positive relationship where t count 6.725> t-table 1.6524. this hypothesis is accepted while the statistical calculation of the advertising intensity variable has a positive effect on student attitudes which are mediated by the level of student motivation. With the R Square value of 0.048, and the Sobel Test result of (5.36045945)> z-absolute 1.96 at the 0.05 level of significance. this is partial mediation. Then for the image variable Polytechnic education has a positive influence on student attitudes which is mediated by the motivation level of students with the value of R Square 0.099, and the Sobel Test shows the number (5.96931448)> z-absolute 1.96 at the 0.05 significance level and is partial mediation. For variables X1 and X2 simultaneously towards the attitude of students to register there is a positive relationship where the value of f count 51,149 is greater than f-table 3.89. this hypothesis is accepted. Suggestions for Ambon State Polytechnic are expected to improve every advertisement submitted on advertising media so that consumers can be interested in registering at the Business Administration Study Program

Student-t Processes as Alternatives to Gaussian Processes

We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process -- a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels -- but has enhanced flexibility, and predictive covariances that, unlike a Gaussian process, explicitly depend on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.Comment: 13 pages, 6 figures, 1 table. To appear in "The Seventeenth International Conference on Artificial Intelligence and Statistics (AISTATS), 2014.