Targeting Bayes factors with direct-path non-equilibrium thermodynamic integration

Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a comparatively small number of parameters, intrinsic errors from sampling fluctuations may outweigh the differences in the log marginal likelihood estimates. In the present article, we propose a thermodynamic integration scheme that directly targets the log Bayes factor. The method is based on a modified annealing path between the posterior distributions of the two models compared, which systematically avoids the high variance prior regime. We combine this scheme with the concept of non-equilibrium TI to minimise discretisation errors from numerical integration. Results obtained on Bayesian regression models applied to standard benchmark data, and a complex hierarchical model applied to biopathway inference, demonstrate a significant reduction in estimator variance over state-of-the-art TI methods

Inference for reaction networks using the Linear Noise Approximation

We consider inference for the reaction rates in discretely observed networks such as those found in models for systems biology, population ecology and epidemics. Most such networks are neither slow enough nor small enough for inference via the true state-dependent Markov jump process to be feasible. Typically, inference is conducted by approximating the dynamics through an ordinary differential equation (ODE), or a stochastic differential equation (SDE). The former ignores the stochasticity in the true model, and can lead to inaccurate inferences. The latter is more accurate but is harder to implement as the transition density of the SDE model is generally unknown. The Linear Noise Approximation (LNA) is a first order Taylor expansion of the approximating SDE about a deterministic solution and can be viewed as a compromise between the ODE and SDE models. It is a stochastic model, but discrete time transition probabilities for the LNA are available through the solution of a series of ordinary differential equations. We describe how a restarting LNA can be efficiently used to perform inference for a general class of reaction networks; evaluate the accuracy of such an approach; and show how and when this approach is either statistically or computationally more efficient than ODE or SDE methods. We apply the LNA to analyse Google Flu Trends data from the North and South Islands of New Zealand, and are able to obtain more accurate short-term forecasts of new flu cases than another recently proposed method, although at a greater computational cost

Universal Codes from Switching Strategies

We discuss algorithms for combining sequential prediction strategies, a task which can be viewed as a natural generalisation of the concept of universal coding. We describe a graphical language based on Hidden Markov Models for defining prediction strategies, and we provide both existing and new models as examples. The models include efficient, parameterless models for switching between the input strategies over time, including a model for the case where switches tend to occur in clusters, and finally a new model for the scenario where the prediction strategies have a known relationship, and where jumps are typically between strongly related ones. This last model is relevant for coding time series data where parameter drift is expected. As theoretical ontributions we introduce an interpolation construction that is useful in the development and analysis of new algorithms, and we establish a new sophisticated lemma for analysing the individual sequence regret of parameterised models

ON THE SMALL SAMPLE PROPERTIES OF DICKEY FULLER AND MAXIMUM LIKELIHOOD UNIT ROOT TESTS ON DISCRETE-SAMPLED SHORT-TERM INTEREST RATES

Testing for unit roots in short-term interest rates plays a key role in the empirical modelling of these series. It is widely assumed that the volatility of interest rates follows some time-varying function which is dependent of the level of the series. This may cause distortions in the performance of conventional tests for unit root nonstationarity since these are typically derived under the assumption of homoskedasticity. Given the relative unfamiliarity on the issue, we conducted an extensive Monte Carlo investigation in order to assess the performance of the DF unit root tests, and examined the effects on the limiting distributions of test procedures (t- and likelihood ratio tests) based on maximum likelihood estimation of models for short-term rates with a linear drift.Unit root, interest rates, CKLS model.

A Variational Autoencoder for Heterogeneous Temporal and Longitudinal Data

The variational autoencoder (VAE) is a popular deep latent variable model used to analyse high-dimensional datasets by learning a low-dimensional latent representation of the data. It simultaneously learns a generative model and an inference network to perform approximate posterior inference. Recently proposed extensions to VAEs that can handle temporal and longitudinal data have applications in healthcare, behavioural modelling, and predictive maintenance. However, these extensions do not account for heterogeneous data (i.e., data comprising of continuous and discrete attributes), which is common in many real-life applications. In this work, we propose the heterogeneous longitudinal VAE (HL-VAE) that extends the existing temporal and longitudinal VAEs to heterogeneous data. HL-VAE provides efficient inference for high-dimensional datasets and includes likelihood models for continuous, count, categorical, and ordinal data while accounting for missing observations. We demonstrate our model's efficacy through simulated as well as clinical datasets, and show that our proposed model achieves competitive performance in missing value imputation and predictive accuracy.Comment: Preprin

On the Small Sample Properties of Dickey Fuller and Maximum Likelihood Unit Root Tests on Discrete-Sampled Short-Term Interest Rates

Testing for unit roots in short-term interest rates plays a key role in the empirical modelling of these series. It is widely assumed that the volatility of interest rates follows some time-varying function which is dependent of the level of the series. This may cause distortions in the performance of conventional tests for unit root nonstationarity since these are typically derived under the assumption of homoskedasticity. Given the relative unfamiliarity on the issue, we conducted an extensive Monte Carlo investigation in order to assess the performance of the DF unit root tests, and examined the effects on the limiting distributions of test procedures (t- and likelihood ratio tests) based on maximum likelihood estimation of models for short-term rates with a linear drift.Unit root, interest rates, CKLS model.