Mixture of normals probit models

This paper generalizes the normal probit model of dichotomous choice by introducing mixtures of normals distributions for the disturbance term. By mixing on both the mean and variance parameters and by increasing the number of distributions in the mixture these models effectively remove the normality assumption and are much closer to semiparametric models. When a Bayesian approach is taken, there is an exact finite-sample distribution theory for the choice probability conditional on the covariates. The paper uses artificial data to show how posterior odds ratios can discriminate between normal and nonnormal distributions in probit models. The method is also applied to female labor force participation decisions in a sample with 1,555 observations from the PSID. In this application, Bayes factors strongly favor mixture of normals probit models over the conventional probit model, and the most favored models have mixtures of four normal distributions for the disturbance term.Econometric models

Log-Regularly Varying Scale Mixture of Normals for Robust Regression

Linear regression with the classical normality assumption for the error distribution may lead to an undesirable posterior inference of regression coefficients due to the potential outliers. This paper considers the finite mixture of two components with thin and heavy tails as the error distribution, which has been routinely employed in applied statistics. For the heavily-tailed component, we introduce the novel class of distributions; their densities are log-regularly varying and have heavier tails than those of Cauchy distribution, yet they are expressed as a scale mixture of normal distributions and enable the efficient posterior inference by Gibbs sampler. We prove the robustness to outliers of the posterior distributions under the proposed models with a minimal set of assumptions, which justifies the use of shrinkage priors with unbounded densities for the coefficient vector in the presence of outliers. The extensive comparison with the existing methods via simulation study shows the improved performance of our model in point and interval estimation, as well as its computational efficiency. Further, we confirm the posterior robustness of our method in the empirical study with the shrinkage priors for regression coefficients.Comment: 62 page

Asymmetric Stochastic Conditional Duration Model --A Mixture of Normals Approach"

This paper extends the stochastic conditional duration model by imposing mixtures of bivariate normal distributions on the innovations of the observation and latent equations of the duration process. This extension allows the model not only to capture the asymmetric behavior of the expected duration but also to easily accommodate a richer dependence structure between the two innovations. In addition, it proposes a novel estimation methodology based on the empirical characteristic function. A set of Monte Carlo experiments as well as empirical applications based on the IBM and Boeing transaction data are provided to assess and illustrate the performance of the proposed model and the estimation method. One main empirical finding in this paper is that there is a signicantly positive "leverage effect" under both the contemporaneous and lagged inter-temporal de pendence structures for the IBM and Boeing duration data.Stochastic Conditional Duration model; Leverage Effect; Discrete Mixtures of Normal; Empirical Characteristic Function

Copula-type Estimators for Flexible Multivariate Density Modeling using Mixtures

Copulas are popular as models for multivariate dependence because they allow the marginal densities and the joint dependence to be modeled separately. However, they usually require that the transformation from uniform marginals to the marginals of the joint dependence structure is known. This can only be done for a restricted set of copulas, e.g. a normal copula. Our article introduces copula-type estimators for flexible multivariate density estimation which also allow the marginal densities to be modeled separately from the joint dependence, as in copula modeling, but overcomes the lack of flexibility of most popular copula estimators. An iterative scheme is proposed for estimating copula-type estimators and its usefulness is demonstrated through simulation and real examples. The joint dependence is is modeled by mixture of normals and mixture of normals factor analyzers models, and mixture of t and mixture of t factor analyzers models. We develop efficient Variational Bayes algorithms for fitting these in which model selection is performed automatically. Based on these mixture models, we construct four classes of copula-type densities which are far more flexible than current popular copula densities, and outperform them in simulation and several real data sets.Comment: 27 pages, 3 figure

On the Bayesian analysis of species sampling mixture models for density estimation

The mixture of normals model has been extensively applied to density estimation problems. This paper proposes an alternative parameterisation that naturally leads to new forms of prior distribution. The parameters can be interpreted as the location, scale and smoothness of the density. Priors on these parameters are often easier to specify. Alternatively, improper and default choices lead to automatic Bayesian density estimation. The ideas are extended to multivariate density estimation

Adaptive Bayesian Estimation of Mixed Discrete-Continuous Distributions under Smoothness and Sparsity

We consider nonparametric estimation of a mixed discrete-continuous distribution under anisotropic smoothness conditions and possibly increasing number of support points for the discrete part of the distribution. For these settings, we derive lower bounds on the estimation rates in the total variation distance. Next, we consider a nonparametric mixture of normals model that uses continuous latent variables for the discrete part of the observations. We show that the posterior in this model contracts at rates that are equal to the derived lower bounds up to a log factor. Thus, Bayesian mixture of normals models can be used for optimal adaptive estimation of mixed discrete-continuous distributions

Flexible Variational Bayes based on a Copula of a Mixture of Normals

Variational Bayes methods approximate the posterior density by a family of tractable distributions and use optimisation to estimate the unknown parameters of the approximation. Variational approximation is useful when exact inference is intractable or very costly. Our article develops a flexible variational approximation based on a copula of a mixture of normals, which is implemented using the natural gradient and a variance reduction method. The efficacy of the approach is illustrated by using simulated and real datasets to approximate multimodal, skewed and heavy-tailed posterior distributions, including an application to Bayesian deep feedforward neural network regression models. Each example shows that the proposed variational approximation is much more accurate than the corresponding Gaussian copula and a mixture of normals variational approximations.Comment: 39 page