Cover Tree Bayesian Reinforcement Learning

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tree method, which remains efficient in high dimensional spaces. We combine the model with Thompson sampling and approximate dynamic programming to obtain effective exploration policies in unknown environments. The flexibility and computational simplicity of the model render it suitable for many reinforcement learning problems in continuous state spaces. We demonstrate this in an experimental comparison with least squares policy iteration

Model selection via Bayesian information capacity designs for generalised linear models

The first investigation is made of designs for screening experiments where the response variable is approximated by a generalised linear model. A Bayesian information capacity criterion is defined for the selection of designs that are robust to the form of the linear predictor. For binomial data and logistic regression, the effectiveness of these designs for screening is assessed through simulation studies using all-subsets regression and model selection via maximum penalised likelihood and a generalised information criterion. For Poisson data and log-linear regression, similar assessments are made using maximum likelihood and the Akaike information criterion for minimally-supported designs that are constructed analytically. The results show that effective screening, that is, high power with moderate type I error rate and false discovery rate, can be achieved through suitable choices for the number of design support points and experiment size. Logistic regression is shown to present a more challenging problem than log-linear regression. Some areas for future work are also indicated

Bayes linear kinematics in the analysis of failure rates and failure time distributions

Collections of related Poisson or binomial counts arise, for example, from a number of different failures in similar machines or neighbouring time periods. A conventional Bayesian analysis requires a rather indirect prior specification and intensive numerical methods for posterior evaluations. An alternative approach using Bayes linear kinematics in which simple conjugate specifications for individual counts are linked through a Bayes linear belief structure is presented. Intensive numerical methods are not required. The use of transformations of the binomial and Poisson parameters is proposed. The approach is illustrated in two examples, one involving a Poisson count of failures, the other involving a binomial count in an analysis of failure times

Active inference and oculomotor pursuit: the dynamic causal modelling of eye movements.

This paper introduces a new paradigm that allows one to quantify the Bayesian beliefs evidenced by subjects during oculomotor pursuit. Subjects' eye tracking responses to a partially occluded sinusoidal target were recorded non-invasively and averaged. These response averages were then analysed using dynamic causal modelling (DCM). In DCM, observed responses are modelled using biologically plausible generative or forward models - usually biophysical models of neuronal activity

Generalisations of Fisher Matrices

Fisher matrices play an important role in experimental design and in data analysis. Their primary role is to make predictions for the inference of model parameters - both their errors and covariances. In this short review, I outline a number of extensions to the simple Fisher matrix formalism, covering a number of recent developments in the field. These are: (a) situations where the data (in the form of (x,y) pairs) have errors in both x and y; (b) modifications to parameter inference in the presence of systematic errors, or through fixing the values of some model parameters; (c) Derivative Approximation for LIkelihoods (DALI) - higher-order expansions of the likelihood surface, going beyond the Gaussian shape approximation; (d) extensions of the Fisher-like formalism, to treat model selection problems with Bayesian evidence.Comment: Invited review article for Entropy special issue on 'Applications of Fisher Information in Sciences'. Accepted versio