Empirical Bayes inference in sparse high-dimensional generalized linear models

Abstract

High-dimensional linear models have been extensively studied in the recent literature, but the developments in high-dimensional generalized linear models, or GLMs, have been much slower. In this paper, we propose the use an empirical or data-driven prior specification leading to an empirical Bayes posterior distribution which can be used for estimation of and inference on the coefficient vector in a high-dimensional GLM, as well as for variable selection. For our proposed method, we prove that the posterior distribution concentrates around the true/sparse coefficient vector at the optimal rate and, furthermore, provide conditions under which the posterior can achieve variable selection consistency. Computation of the proposed empirical Bayes posterior is simple and efficient, and, in terms of variable selection in logistic and Poisson regression, is shown to perform well in simulations compared to existing Bayesian and non-Bayesian methods.Comment: 30 pages, 2 table