Bayesian Semiparametric Regression

Abstract: We consider Bayesian estimation of restricted conditional moment models with linear regression as a particular example. The standard practice in the Bayesian literature for semiparametric models is to use flexible families of distributionsfor the errors and assume that the errors are independent from covariates. However, a model with flexible covariate dependent error distributions should be preferred for the following reasons: consistent estimation of the parameters of interest evenif errors and covariates are dependent; possibly superior prediction intervals and more efficient estimation of the parameters under heteroscedasticity. To address these issues, we develop a Bayesian semiparametric model with flexible predictor dependent error densities and with mean restricted by a conditional moment condition. Sufficient conditions to achieve posterior consistency of the regression parameters and conditional error densities are provided. In experiments, the proposed method compares favorably with classical and alternative Bayesian estimation methods for the estimation of the regression coefficients.

Modeling Probabilities of Patent Oppositions in a Bayesian Semiparametric Regression Framework

Most econometric analyses of patent data rely on regression methods using a parametric form of the predictor for modeling the dependence of the response given certain covariates. These methods often lack the capability of identifying non-linear relationships between dependent and independent variables. We present an approach based on a generalized additive model in order to avoid these shortcomings. Our method is fully Bayesian and makes use of Markov Chain Monte Carlo (MCMC) simulation techniques for estimation purposes. Using this methodology we reanalyze the determinants of patent oppositions in Europe for biotechnology/pharmaceutical and semiconductor/computer software patents. Our results largely confirm the findings of a previous parametric analysis of the same data provided by Graham, Hall, Harhoff&Mowery (2002). However, our model specification clearly verifies considerable non-linearities in the effect of various metrical covariates on the probability of an opposition. Furthermore, our semiparametric approach shows that the categorizations of these covariates made by Graham et al. (2002) cannot capture those non--linearities and, from a statistical point of view, appear to somehow ad hoc

Bayesian Semiparametric Regression Analysis of Multicategorical Time-Space Data

We present a unified semiparametric Bayesian approach based on Markov random field priors for analyzing the dependence of multicategorical response variables on time, space and further covariates. The general model extends dynamic, or state space, models for categorical time series and longitudinal data by including spatial effects as well as nonlinear effects of metrical covariates in flexible semiparametric form. Trend and seasonal components, different types of covariates and spatial effects are all treated within the same general framework by assigning appropriate priors with different forms and degrees of smoothness. Inference is fully Bayesian and uses MCMC techniques for posterior analysis. We provide two approaches: The first one is based on direct evaluation of observation likelihoods. The second one is based on latent semiparametric utility models and is particularly useful for probit models. The methods are illustrated by applications to unemployment data and a forest damage survey

A Note on Bayes Semiparametric Regression

In the Bayesian approach to inference, all unknown quantities contained in a probability model for the observed data are treated as random variables. Specifically, the fixed but unknown parameters are viewed as random variables under the Bayesian approach. In this paper, Bayesian approach is employed to making inferences on the semiparametric regression model as mixed model , and we prove some theorems about posterior. Keywords?Mixed models, Semiparametric regression, Penalized spline, Bayesian inference, Prior density, Posterior density

Semiparametric Bayesian inference in multiple equation models

This paper outlines an approach to Bayesian semiparametric regression in multiple equation models which can be used to carry out inference in seemingly unrelated regressions or simultaneous equations models with nonparametric components. The approach treats the points on each nonparametric regression line as unknown parameters and uses a prior on the degree of smoothness of each line to ensure valid posterior inference despite the fact that the number of parameters is greater than the number of observations. We develop an empirical Bayesian approach that allows us to estimate the prior smoothing hyperparameters from the data. An advantage of our semiparametric model is that it is written as a seemingly unrelated regressions model with independent normal-Wishart prior. Since this model is a common one, textbook results for posterior inference, model comparison, prediction and posterior computation are immediately available. We use this model in an application involving a two-equation structural model drawn from the labour and returns to schooling literatures

Variational inference for count response semiparametric regression

© 2015 International Society for Bayesian Analysis. Fast variational approximate algorithms are developed for Bayesian semiparametric regression when the response variable is a count, i.e., a nonnegative integer. We treat both the Poisson and Negative Binomial families as models for the response variable. Our approach utilizes recently developed methodology known as non-conjugate variational message passing. For concreteness, we focus on generalized additive mixed models, although our variational approximation approach extends to a wide class of semiparametric regression models such as those containing interactions and elaborate random effect structure

Semiparametric regression analysis via Infer.NET

© 2018, American Statistical Association. All rights reserved. We provide several examples of Bayesian semiparametric regression analysis via the Infer.NET package for approximate deterministic inference in Bayesian models. The examples are chosen to encompass a wide range of semiparametric regression situations. Infer.NET is shown to produce accurate inference in comparison with Markov chain Monte Carlo via the BUGS package, but to be considerably faster. Potentially, this contribution represents the start of a new era for semiparametric regression, where large and complex analyses are performed via fast Bayesian inference methodology and software, mainly being developed within Machine Learning

Scalable Bayesian nonparametric regression via a Plackett-Luce model for conditional ranks

We present a novel Bayesian nonparametric regression model for covariates X and continuous, real response variable Y. The model is parametrized in terms of marginal distributions for Y and X and a regression function which tunes the stochastic ordering of the conditional distributions F(y|x). By adopting an approximate composite likelihood approach, we show that the resulting posterior inference can be decoupled for the separate components of the model. This procedure can scale to very large datasets and allows for the use of standard, existing, software from Bayesian nonparametric density estimation and Plackett-Luce ranking estimation to be applied. As an illustration, we show an application of our approach to a US Census dataset, with over 1,300,000 data points and more than 100 covariates