3,493 research outputs found
Particle Gibbs for Bayesian Additive Regression Trees
Additive regression trees are flexible non-parametric models and popular
off-the-shelf tools for real-world non-linear regression. In application
domains, such as bioinformatics, where there is also demand for probabilistic
predictions with measures of uncertainty, the Bayesian additive regression
trees (BART) model, introduced by Chipman et al. (2010), is increasingly
popular. As data sets have grown in size, however, the standard
Metropolis-Hastings algorithms used to perform inference in BART are proving
inadequate. In particular, these Markov chains make local changes to the trees
and suffer from slow mixing when the data are high-dimensional or the best
fitting trees are more than a few layers deep. We present a novel sampler for
BART based on the Particle Gibbs (PG) algorithm (Andrieu et al., 2010) and a
top-down particle filtering algorithm for Bayesian decision trees
(Lakshminarayanan et al., 2013). Rather than making local changes to individual
trees, the PG sampler proposes a complete tree to fit the residual. Experiments
show that the PG sampler outperforms existing samplers in many settings
Bayesian Additive Regression Trees with Model Trees
Bayesian Additive Regression Trees (BART) is a tree-based machine learning
method that has been successfully applied to regression and classification
problems. BART assumes regularisation priors on a set of trees that work as
weak learners and is very flexible for predicting in the presence of
non-linearity and high-order interactions. In this paper, we introduce an
extension of BART, called Model Trees BART (MOTR-BART), that considers
piecewise linear functions at node levels instead of piecewise constants. In
MOTR-BART, rather than having a unique value at node level for the prediction,
a linear predictor is estimated considering the covariates that have been used
as the split variables in the corresponding tree. In our approach, local
linearities are captured more efficiently and fewer trees are required to
achieve equal or better performance than BART. Via simulation studies and real
data applications, we compare MOTR-BART to its main competitors. R code for
MOTR-BART implementation is available at https://github.com/ebprado/MOTR-BART
Tree models: a Bayesian perspective
Submitted in partial fulfilment of the requirements for the degree of Master of Philosophy at Queen Mary, University of London, November 2006Classical tree models represent an attempt to create nonparametric models which
have good predictive powers as well a simple structure readily comprehensible by non-
experts. Bayesian tree models have been created by a team consisting of Chipman,
George and McCulloch and second team consisting of Denison, Mallick and Smith.
Both approaches employ Green's Reversible Jump Markov Chain Monte Carlo tech-
nique to carry out a more e®ective search than the `greedy' methods used classically.
The aim of this work is to evaluate both types of Bayesian tree models from a
Bayesian perspective and compare them
- …