Estimation of parameters in linear structural relationships: Sensitivity to the choice of the ratio of error variances

Maximum likelihood estimation of parameters in linear structural relationships under normality assumptions requires knowledge of one or more of the model parameters if no replication is available. The most common assumption added to the model definition is that the ratio of the error variances of the response and predictor variates is known. The use of asymptotic formulae for variances and mean squared errors as a function of sample size and the assumed value for the error variance ratio is investigated

Particle Gibbs for Bayesian Additive Regression Trees

Additive regression trees are flexible non-parametric models and popular off-the-shelf tools for real-world non-linear regression. In application domains, such as bioinformatics, where there is also demand for probabilistic predictions with measures of uncertainty, the Bayesian additive regression trees (BART) model, introduced by Chipman et al. (2010), is increasingly popular. As data sets have grown in size, however, the standard Metropolis-Hastings algorithms used to perform inference in BART are proving inadequate. In particular, these Markov chains make local changes to the trees and suffer from slow mixing when the data are high-dimensional or the best fitting trees are more than a few layers deep. We present a novel sampler for BART based on the Particle Gibbs (PG) algorithm (Andrieu et al., 2010) and a top-down particle filtering algorithm for Bayesian decision trees (Lakshminarayanan et al., 2013). Rather than making local changes to individual trees, the PG sampler proposes a complete tree to fit the residual. Experiments show that the PG sampler outperforms existing samplers in many settings

Mondrian Forests for Large-Scale Regression when Uncertainty Matters

Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to large-scale data sets comes at the cost of approximating the uncertainty estimates. We extend Mondrian forests, first proposed by Lakshminarayanan et al. (2014) for classification problems, to the large-scale non-parametric regression setting. Using a novel hierarchical Gaussian prior that dovetails with the Mondrian forest framework, we obtain principled uncertainty estimates, while still retaining the computational advantages of decision forests. Through a combination of illustrative examples, real-world large-scale datasets, and Bayesian optimization benchmarks, we demonstrate that Mondrian forests outperform approximate GPs on large-scale regression tasks and deliver better-calibrated uncertainty assessments than decision-forest-based methods.Comment: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS) 2016, Cadiz, Spain. JMLR: W&CP volume 5

The Evolution of Our Preferences: Evidence from Capuchin-Monkey Trading Behavior

Behavioral economics has demonstrated systematic decision-making biases in both lab and field data. But are these biases learned or innate? We investigate this question using experiments on a novel set of subjects — capuchin monkeys. By introducing a fiat currency and trade to a capuchin colony, we are able to recover their preferences over a wide range of goods and risky choices. We show that standard price theory does a remarkably good job of describing capuchin purchasing behavior; capuchin monkeys react rationally to both price and wealth shocks. However, when capuchins are faced with more complex choices including risky gambles, they display many of the hallmark biases of human behavior, including reference-dependent choices and loss-aversion. Given that capuchins demonstrate little to no social learning and lack experience with abstract gambles, these results suggest that certain biases such as loss-aversion are an innate function of how our brains code experiences, rather than learned behavior or the result of misapplied heuristics.Prospect theory, Loss aversion, Reference dependence, Evolution, Neuroeconomics, Capuchin monkeys, Monkey business

Training Big Random Forests with Little Resources

Without access to large compute clusters, building random forests on large datasets is still a challenging problem. This is, in particular, the case if fully-grown trees are desired. We propose a simple yet effective framework that allows to efficiently construct ensembles of huge trees for hundreds of millions or even billions of training instances using a cheap desktop computer with commodity hardware. The basic idea is to consider a multi-level construction scheme, which builds top trees for small random subsets of the available data and which subsequently distributes all training instances to the top trees' leaves for further processing. While being conceptually simple, the overall efficiency crucially depends on the particular implementation of the different phases. The practical merits of our approach are demonstrated using dense datasets with hundreds of millions of training instances.Comment: 9 pages, 9 Figure

Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages

We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classical EP, which may not have an analytic expression. We use kernel-based regression, which is trained on a set of probability distributions representing the incoming messages, and the associated outgoing messages. The kernel approach has two main advantages: first, it is fast, as it is implemented using a novel two-layer random feature representation of the input message distributions; second, it has principled uncertainty estimates, and can be cheaply updated online, meaning it can request and incorporate new training data when it encounters inputs on which it is uncertain. In experiments, our approach is able to solve learning problems where a single message operator is required for multiple, substantially different data sets (logistic regression for a variety of classification problems), where it is essential to accurately assess uncertainty and to efficiently and robustly update the message operator.Comment: accepted to UAI 2015. Correct typos. Add more content to the appendix. Main results unchange

The Evolution of Our Preferences: Evidence from Capuchin-Monkey Trading Behavior

Behavioral economics has demonstrated systematic decision-making biases in both lab and field data. But are these biases learned or innate? We investigate this question using experiments on a novel set of subjects — capuchin monkeys. By introducing a fiat currency and trade to a capuchin colony, we are able to recover their preferences over a wide range of goods and risky choices. We show that standard price theory does a remarkably good job of describing capuchin purchasing behavior; capuchin monkeys react rationally to both price and wealth shocks. However, when capuchins are faced with more complex choices including risky gambles, they display many of the hallmark biases of human behavior, including reference-dependent choices and loss-aversion. Given that capuchins demonstrate little to no social learning and lack experience with abstract gambles, these results suggest that certain biases such as loss-aversion are an innate function of how our brains code experiences, rather than learned behavior or the result of misapplied heuristics