10,015 research outputs found
Bayesian Nonparametric Estimation and Consistency of Mixed Multinomial Logit Choice Models
This paper develops nonparametric estimation for discrete choice models based on the Mixed Multinomial Logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived under the assumption of random utility maximization, subject to the identification of an unknown distribution G. Noting the mixture model description of the MMNL, we employ a Bayesian nonparametric approach, using nonparametric priors on the unknown mixing distribution G, to estimate the unknown choice probabilities. Theoretical support for the use of the proposed methodology is provided by establishing strong consistency of a general nonparametric prior on G under simple sufficient conditions. Consistency is defined according to a L1-type distance on the space of choice probabilities and is achieved by extending to a regression model framework a recent approach to strong consistency based on the summability of square roots of prior probabilities. Moving to estimation, slightly different techniques for non-panel and panel data models are discussed. For practical implementation, we describe efficient and relatively easy to use blocked Gibbs sampling procedures. A simulation study is also performed to illustrate the proposed methods and the
exibility they achieve with respect to parametric Gaussian MMNL models.Bayesian consistency, Bayesian nonparametrics, Blocked Gibbs sampler, Discrete choice models, Mixed Multinomial Logit, Random probability measures, Stick-breaking priors
Bayesian nonparametric estimation and consistency of mixed multinomial logit choice models
This paper develops nonparametric estimation for discrete choice models based
on the mixed multinomial logit (MMNL) model. It has been shown that MMNL models
encompass all discrete choice models derived under the assumption of random
utility maximization, subject to the identification of an unknown distribution
. Noting the mixture model description of the MMNL, we employ a Bayesian
nonparametric approach, using nonparametric priors on the unknown mixing
distribution , to estimate choice probabilities. We provide an important
theoretical support for the use of the proposed methodology by investigating
consistency of the posterior distribution for a general nonparametric prior on
the mixing distribution. Consistency is defined according to an -type
distance on the space of choice probabilities and is achieved by extending to a
regression model framework a recent approach to strong consistency based on the
summability of square roots of prior probabilities. Moving to estimation,
slightly different techniques for non-panel and panel data models are
discussed. For practical implementation, we describe efficient and relatively
easy-to-use blocked Gibbs sampling procedures. These procedures are based on
approximations of the random probability measure by classes of finite
stick-breaking processes. A simulation study is also performed to investigate
the performance of the proposed methods.Comment: Published in at http://dx.doi.org/10.3150/09-BEJ233 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Bayesian Approximate Kernel Regression with Variable Selection
Nonlinear kernel regression models are often used in statistics and machine
learning because they are more accurate than linear models. Variable selection
for kernel regression models is a challenge partly because, unlike the linear
regression setting, there is no clear concept of an effect size for regression
coefficients. In this paper, we propose a novel framework that provides an
effect size analog of each explanatory variable for Bayesian kernel regression
models when the kernel is shift-invariant --- for example, the Gaussian kernel.
We use function analytic properties of shift-invariant reproducing kernel
Hilbert spaces (RKHS) to define a linear vector space that: (i) captures
nonlinear structure, and (ii) can be projected onto the original explanatory
variables. The projection onto the original explanatory variables serves as an
analog of effect sizes. The specific function analytic property we use is that
shift-invariant kernel functions can be approximated via random Fourier bases.
Based on the random Fourier expansion we propose a computationally efficient
class of Bayesian approximate kernel regression (BAKR) models for both
nonlinear regression and binary classification for which one can compute an
analog of effect sizes. We illustrate the utility of BAKR by examining two
important problems in statistical genetics: genomic selection (i.e. phenotypic
prediction) and association mapping (i.e. inference of significant variants or
loci). State-of-the-art methods for genomic selection and association mapping
are based on kernel regression and linear models, respectively. BAKR is the
first method that is competitive in both settings.Comment: 22 pages, 3 figures, 3 tables; theory added; new simulations
presented; references adde
Generalized structured additive regression based on Bayesian P-splines
Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM's and extensions to generalized structured additive regression based on one or two dimensional P-splines as the main building block. The approach extends previous work by Lang und Brezger (2003) for Gaussian responses. Inference relies on Markov chain Monte Carlo (MCMC) simulation techniques, and is either based on iteratively weighted least squares (IWLS) proposals or on latent utility representations of (multi)categorical regression models. Our approach covers the most common univariate response distributions, e.g. the Binomial, Poisson or Gamma distribution, as well as multicategorical responses. For the first time, we present Bayesian semiparametric inference for the widely used multinomial logit models. As we will demonstrate through two applications on the forest health status of trees and a space-time analysis of health insurance data, the approach allows realistic modelling of complex problems. We consider the enormous flexibility and extendability of our approach as a main advantage of Bayesian inference based on MCMC techniques compared to more traditional approaches. Software for the methodology presented in the paper is provided within the public domain package BayesX
Bayesian learning of joint distributions of objects
There is increasing interest in broad application areas in defining flexible
joint models for data having a variety of measurement scales, while also
allowing data of complex types, such as functions, images and documents. We
consider a general framework for nonparametric Bayes joint modeling through
mixture models that incorporate dependence across data types through a joint
mixing measure. The mixing measure is assigned a novel infinite tensor
factorization (ITF) prior that allows flexible dependence in cluster allocation
across data types. The ITF prior is formulated as a tensor product of
stick-breaking processes. Focusing on a convenient special case corresponding
to a Parafac factorization, we provide basic theory justifying the flexibility
of the proposed prior and resulting asymptotic properties. Focusing on ITF
mixtures of product kernels, we develop a new Gibbs sampling algorithm for
routine implementation relying on slice sampling. The methods are compared with
alternative joint mixture models based on Dirichlet processes and related
approaches through simulations and real data applications.Comment: Appearing in Proceedings of the 16th International Conference on
Artificial Intelligence and Statistics (AISTATS) 2013, Scottsdale, AZ, US
Locally Adaptive Function Estimation for Binary Regression Models
In this paper we present a nonparametric Bayesian approach for fitting unsmooth or highly oscillating functions in regression models with binary responses. The approach extends previous work by Lang et al. (2002) for Gaussian responses. Nonlinear functions are modelled by first or second order random walk priors with locally varying variances or smoothing parameters. Estimation is fully Bayesian and uses latent utility representations of binary regression models for efficient block sampling from the full conditionals of nonlinear functions
- ā¦