6,807 research outputs found
Random projections for Bayesian regression
This article deals with random projections applied as a data reduction
technique for Bayesian regression analysis. We show sufficient conditions under
which the entire -dimensional distribution is approximately preserved under
random projections by reducing the number of data points from to in the case . Under mild
assumptions, we prove that evaluating a Gaussian likelihood function based on
the projected data instead of the original data yields a
-approximation in terms of the Wasserstein
distance. Our main result shows that the posterior distribution of Bayesian
linear regression is approximated up to a small error depending on only an
-fraction of its defining parameters. This holds when using
arbitrary Gaussian priors or the degenerate case of uniform distributions over
for . Our empirical evaluations involve different
simulated settings of Bayesian linear regression. Our experiments underline
that the proposed method is able to recover the regression model up to small
error while considerably reducing the total running time
Bayesian Regression of Piecewise Constant Functions
We derive an exact and efficient Bayesian regression algorithm for piecewise
constant functions of unknown segment number, boundary location, and levels. It
works for any noise and segment level prior, e.g. Cauchy which can handle
outliers. We derive simple but good estimates for the in-segment variance. We
also propose a Bayesian regression curve as a better way of smoothing data
without blurring boundaries. The Bayesian approach also allows straightforward
determination of the evidence, break probabilities and error estimates, useful
for model selection and significance and robustness studies. We discuss the
performance on synthetic and real-world examples. Many possible extensions will
be discussed.Comment: 27 pages, 18 figures, 1 table, 3 algorithm
Robust Bayesian Regression with Synthetic Posterior
Although linear regression models are fundamental tools in statistical
science, the estimation results can be sensitive to outliers. While several
robust methods have been proposed in frequentist frameworks, statistical
inference is not necessarily straightforward. We here propose a Bayesian
approach to robust inference on linear regression models using synthetic
posterior distributions based on -divergence, which enables us to
naturally assess the uncertainty of the estimation through the posterior
distribution. We also consider the use of shrinkage priors for the regression
coefficients to carry out robust Bayesian variable selection and estimation
simultaneously. We develop an efficient posterior computation algorithm by
adopting the Bayesian bootstrap within Gibbs sampling. The performance of the
proposed method is illustrated through simulation studies and applications to
famous datasets.Comment: 23 pages, 5 figure
On the Convergence of Bayesian Regression Models
We consider heteroscedastic nonparametric regression models, when both the
mean function and variance function are unknown and to be estimated with
nonparametric approaches. We derive convergence rates of posterior
distributions for this model with different priors, including splines and
Gaussian process priors. The results are based on the general ones on the rates
of convergence of posterior distributions for independent, non-identically
distributed observations, and are established for both of the cases with random
covariates, and deterministic covariates. We also illustrate that the results
can be achieved for all levels of regularity, which means they are adaptive
Bayesian Regression Markets
Machine learning tasks are vulnerable to the quality of data used as input.
Yet, it is often challenging for firms to obtain adequate datasets, with them
being naturally distributed amongst owners, that in practice, may be
competitors in a downstream market and reluctant to share information. Focusing
on supervised learning for regression tasks, we develop a \textit{regression
market} to provide a monetary incentive for data sharing. Our proposed
mechanism adopts a Bayesian framework, allowing us to consider a more general
class of regression tasks. We present a thorough exploration of the market
properties, and show that similar proposals in current literature expose the
market agents to sizeable financial risks, which can be mitigated in our
probabilistic setting.Comment: 46 pages, 11 figures, 2 table
Bayesian regression analysis.
Regression analysis is a statistical method used to relate a variable of interest, typically y (the dependent variable), to a set of independent variables, usually, X1, X2,...,Xn . The goal is to build a model that assists statisticians in describing, controlling, and predicting the dependent variable based on the independent variable(s). There are many types of regression analysis: Simple and Multiple Linear Regression, Nonlinear Regression, and Bayesian Regression Analysis to name a few. Here we will explore simple and multiple linear regression and Bayesian linear regression. For years, the most widely used method of regression analysis has been the Frequentist methods, or simple and multiple regression. However, with the advancements of computers and computing tools such as WinBUGS, Bayesian methods have become more widely accepted. With the use of WinBUGS, we can utilize a Markov Chain Monte Carlo (MCMC) method called Gibbs Sampling to simplify the increasingly difficult calculations. Given that Bayesian regression analysis is a relatively new method, it is not without faults. Many in the statistical community find that the use of Bayesian techniques is not a satisfactory method since the choice of the prior distribution is purely a guessing game and varies from statistician to statistician. In this thesis, an example is presented using both Frequentist and Bayesian methods and a comparison is made between the two. As computers become more advanced, the use of Bayesian regression analysis may become more widely accepted as the method of choice for regression analyses as it allows for the interpretation of a probability as a measure of degree of belief concerning actual data observed
- …