On nonparametric estimation of a mixing density via the predictive recursion algorithm

Nonparametric estimation of a mixing density based on observations from the corresponding mixture is a challenging statistical problem. This paper surveys the literature on a fast, recursive estimator based on the predictive recursion algorithm. After introducing the algorithm and giving a few examples, I summarize the available asymptotic convergence theory, describe an important semiparametric extension, and highlight two interesting applications. I conclude with a discussion of several recent developments in this area and some open problems.Comment: 22 pages, 5 figures. Comments welcome at https://www.researchers.one/article/2018-12-

Semiparametric inference in mixture models with predictive recursion marginal likelihood

Predictive recursion is an accurate and computationally efficient algorithm for nonparametric estimation of mixing densities in mixture models. In semiparametric mixture models, however, the algorithm fails to account for any uncertainty in the additional unknown structural parameter. As an alternative to existing profile likelihood methods, we treat predictive recursion as a filter approximation to fitting a fully Bayes model, whereby an approximate marginal likelihood of the structural parameter emerges and can be used for inference. We call this the predictive recursion marginal likelihood. Convergence properties of predictive recursion under model mis-specification also lead to an attractive construction of this new procedure. We show pointwise convergence of a normalized version of this marginal likelihood function. Simulations compare the performance of this new marginal likelihood approach that of existing profile likelihood methods as well as Dirichlet process mixtures in density estimation. Mixed-effects models and an empirical Bayes multiple testing application in time series analysis are also considered

Appropriate use of parametric and nonparametric methods in estimating regression models with various shapes of errors

In this paper, a practical estimation method for a regression model is proposed using semiparametric efficient score functions applicable to data with various shapes of errors. First, I derive semiparametric efficient score vectors for a homoscedastic regression model without any assumptions of errors. Next, the semiparametric efficient score function can be modified assuming a certain parametric distribution of errors according to the shape of the error distribution or by estimating the error distribution non-parametrically. Nonparametric methods for errors can be used to estimate the parameters of interest or to find an appropriate parametric error distribution. In this regard, the proposed estimation methods utilize both parametric and nonparametric methods for errors appropriately. Through numerical studies, the performance of the proposed estimation methods is demonstrated.Comment: 2 figure

A Survey of Recent Theoretical Results for Time Series Models with GARCH Errors,

This paper provides a review of some recent theoretical results for time series models with GARCH errors, and is directed towards practitioners. Starting with the simple ARCH model and proceeding to the GARCH model, some results for stationary and nonstationary ARMA-GARCH are summarized. Various new ARCH-type models, including double threshold ARCH and GARCH, ARFIMA-GARCH, CHARMA and vector ARMA-GARCH, are also reviewed.

A PRticle filter algorithm for nonparametric estimation of multivariate mixing distributions

Predictive recursion (PR) is a fast, recursive algorithm that gives a smooth estimate of the mixing distribution under the general mixture model. However, the PR algorithm requires evaluation of a normalizing constant at each iteration. When the support of the mixing distribution is of relatively low dimension, this is not a problem since quadrature methods can be used and are very efficient. But when the support is of higher dimension, quadrature methods are inefficient and there is no obvious Monte Carlo-based alternative. In this paper, we propose a new strategy, which we refer to as PRticle filter, wherein we augment the basic PR algorithm with a filtering mechanism that adaptively reweights an initial set of particles along the updating sequence which are used to obtain Monte Carlo approximations of the normalizing constants. Convergence properties of the PRticle filter approximation are established and its empirical accuracy is demonstrated with simulation studies and a marked spatial point process data analysis.Comment: 21 pages, 5 figure

Estimating a mixing distribution on the sphere using predictive recursion

Mixture models are commonly used when data show signs of heterogeneity and, often, it is important to estimate the distribution of the latent variable responsible for that heterogeneity. This is a common problem for data taking values in a Euclidean space, but the work on mixing distribution estimation based on directional data taking values on the unit sphere is limited. In this paper, we propose using the predictive recursion (PR) algorithm to solve for a mixture on a sphere. One key feature of PR is its computational efficiency. Moreover, compared to likelihood-based methods that only support finite mixing distribution estimates, PR is able to estimate a smooth mixing density. PR's asymptotic consistency in spherical mixture models is established, and simulation results showcase its benefits compared to existing likelihood-based methods. We also show two real-data examples to illustrate how PR can be used for goodness-of-fit testing and clustering.Comment: 26 pages, 8 figures, 2 table

Revisiting consistency of a recursive estimator of mixing distributions

Estimation of the mixing distribution under a general mixture model is a very difficult problem, especially when the mixing distribution is assumed to have a density. Predictive recursion (PR) is a fast, recursive algorithm for nonparametric estimation of a mixing distribution/density in general mixture models. However, the existing PR consistency results make rather strong assumptions, some of which fail for a class of mixture models relevant for monotone density estimation, namely, scale mixtures of uniform kernels. In this paper, we develop new consistency results for PR under weaker conditions. Armed with this new theory, we prove that PR is consistent for the scale mixture of uniforms problem, and we show that the corresponding PR mixture density estimator has very good practical performance compared to several existing methods for monotone density estimation.Comment: 27 pages, 3 figure