Estimation in Dirichlet random effects models

We develop a new Gibbs sampler for a linear mixed model with a Dirichlet process random effect term, which is easily extended to a generalized linear mixed model with a probit link function. Our Gibbs sampler exploits the properties of the multinomial and Dirichlet distributions, and is shown to be an improvement, in terms of operator norm and efficiency, over other commonly used MCMC algorithms. We also investigate methods for the estimation of the precision parameter of the Dirichlet process, finding that maximum likelihood may not be desirable, but a posterior mode is a reasonable approach. Examples are given to show how these models perform on real data. Our results complement both the theoretical basis of the Dirichlet process nonparametric prior and the computational work that has been done to date.Comment: Published in at http://dx.doi.org/10.1214/09-AOS731 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Binary and Ordinal Random Effects Models Including Variable Selection

A likelihood-based boosting approach for fitting binary and ordinal mixed models is presented. In contrast to common procedures it can be used in high-dimensional settings where a large number of potentially influential explanatory variables is available. Constructed as a componentwise boosting method it is able to perform variable selection with the complexity of the resulting estimator being determined by information criteria. The method is investigated in simulation studies both for cumulative and sequential models and is illustrated by using real data sets

Asymptotics for random effects models with serial correlation

This paper considers the large sample behavior of the maximum likelihood estimator of random effects models. Consistent estimation and asymptotic normality as N and/or T grows large is established for a comprehensive specification which allows for serial correlation in the form of AR(1) for the idiosyncratic or time-specific error component. The consistency and asymptotic normality properties of all commonly used random effects models are obtained as special cases of the comprehensive model. When N or T \rightarrow \infty only a subset of the parameters are consistent and asymptotic normality is established for the consistent subsets.Panel data; error components; consistency; asymptotic normality; maximum likelihood.

Scalable inference for crossed random effects models

We develop methodology and complexity theory for Markov chain Monte Carlo algorithms used in inference for crossed random effect models in modern analysis of variance. We consider 15 a plain Gibbs sampler and a simple modification we propose here, a collapsed Gibbs sampler. Under some balancedness assumptions on the data designs and assuming that precision hyperparameters are known, we demonstrate that the plain Gibbs sampler is not scalable, in the sense that its complexity is worse than proportional to the number of parameters and data, but that the collapsed Gibbs sampler is scalable. In simulated and real datasets we show that the explicit 20 convergence rates our theory predicts match remarkably the computable but non-explicit rates in cases where the design assumptions are violated. We also show empirically that the collapsed Gibbs sampler, extended to sample precision hyperparameters, outperforms significantly, often by orders of magnitude, alternative state of the art algorithms. Supplementary material includes some proofs, additional simulations, implementation details and the R code to implement the 25 algorithms considered in the article

Asymptotic properties of the maximum likelihood estimator of random effects models with serial correlation

This paper considers the large sample behavior of the maximum likelihood estimator of random effects models with serial correlation in the form of AR(1) for the idiosyncratic or time-specific error component. Consistent estimation and asymptotic normality as N and/or T grows large is established for a comprehensive specification which nests these models as well as all commonly used random effects models. When only N or T grows large only a subset of the parameters are consistent and asymptotic normality is established for the consistent subsets.Panel data; serial correlation; random effects