Sequential Gaussian Processes for Online Learning of Nonstationary Functions

Many machine learning problems can be framed in the context of estimating functions, and often these are time-dependent functions that are estimated in real-time as observations arrive. Gaussian processes (GPs) are an attractive choice for modeling real-valued nonlinear functions due to their flexibility and uncertainty quantification. However, the typical GP regression model suffers from several drawbacks: i) Conventional GP inference scales $O(N^{3})$ with respect to the number of observations; ii) updating a GP model sequentially is not trivial; and iii) covariance kernels often enforce stationarity constraints on the function, while GPs with non-stationary covariance kernels are often intractable to use in practice. To overcome these issues, we propose an online sequential Monte Carlo algorithm to fit mixtures of GPs that capture non-stationary behavior while allowing for fast, distributed inference. By formulating hyperparameter optimization as a multi-armed bandit problem, we accelerate mixing for real time inference. Our approach empirically improves performance over state-of-the-art methods for online GP estimation in the context of prediction for simulated non-stationary data and hospital time series data

From nominal to true a posteriori probabilities: an exact Bayesian theorem based probabilistic data association approach for iterative MIMO detection and decoding

It was conventionally regarded that the existing probabilistic data association (PDA) algorithms output the estimated symbol-wise a posteriori probabilities (APPs) as soft information. In this paper, however, we demonstrate that these probabilities are not the true APPs in the rigorous mathematicasense, but a type of nominal APPs, which are unsuitable for the classic architecture of iterative detection and decoding (IDD) aided receivers. To circumvent this predicament, we propose an exact Bayesian theorem based logarithmic domain PDA (EB-Log-PDA) method, whose output has similar characteristics to the true APPs, and hence it is readily applicable to the classic IDD architecture of multiple-input multiple-output (MIMO) systems using the general M-ary modulation. Furthermore, we investigate the impact of the PDA algorithms' inner iteration on the design of PDA-aided IDD receivers. We demonstrate that introducing inner iterations into PDAs, which is common practice in PDA-aided uncoded MIMO systems, would actually degrade the IDD receiver's performance, despite significantly increasing the overall computational complexity of the IDD receiver. Finally, we investigate the relationship between the extrinsic log-likelihood ratio (LLRs) of the proposed EB-Log-PDA and of the approximate Bayesian theorem based logarithmic domain PDA (AB-Log-PDA) reported in our previous work. We also show that the IDD scheme employing the EB-Log-PDA without incorporating any inner PDA iterations has an achievable performance close to that of the optimal maximum a posteriori (MAP) detector based IDD receiver, while imposing a significantly lower computational complexity in the scenarios considered

Isomorphic classical molecular dynamics model for an excess electron in a supercritical fluid

Ring polymer molecular dynamics (RPMD) is used to directly simulate the dynamics of an excess electron in a supercritical fluid over a broad range of densities. The accuracy of the RPMD model is tested against numerically exact path integral statistics through the use of analytical continuation techniques. At low fluid densities, the RPMD model substantially underestimates the contribution of delocalized states to the dynamics of the excess electron. However, with increasing solvent density, the RPMD model improves, nearly satisfying analytical continuation constraints at densities approaching those of typical liquids. In the high density regime, quantum dispersion substantially decreases the self-diffusion of the solvated electron. In this regime where the dynamics of the electron is strongly coupled to the dynamics of the atoms in the fluid, trajectories that can reveal diffusive motion of the electron are long in comparison to $\beta\hbar$.Comment: 24 pages, 4 figure

Fast Covariance Estimation for High-dimensional Functional Data

For smoothing covariance functions, we propose two fast algorithms that scale linearly with the number of observations per function. Most available methods and software cannot smooth covariance matrices of dimension $J \times J$ with $J>500$; the recently introduced sandwich smoother is an exception, but it is not adapted to smooth covariance matrices of large dimensions such as $J \ge 10,000$. Covariance matrices of order $J=10,000$, and even $J=100,000$, are becoming increasingly common, e.g., in 2- and 3-dimensional medical imaging and high-density wearable sensor data. We introduce two new algorithms that can handle very large covariance matrices: 1) FACE: a fast implementation of the sandwich smoother and 2) SVDS: a two-step procedure that first applies singular value decomposition to the data matrix and then smoothes the eigenvectors. Compared to existing techniques, these new algorithms are at least an order of magnitude faster in high dimensions and drastically reduce memory requirements. The new algorithms provide instantaneous (few seconds) smoothing for matrices of dimension $J=10,000$ and very fast ($<$ 10 minutes) smoothing for $J=100,000$. Although SVDS is simpler than FACE, we provide ready to use, scalable R software for FACE. When incorporated into R package {\it refund}, FACE improves the speed of penalized functional regression by an order of magnitude, even for data of normal size ($J <500$). We recommend that FACE be used in practice for the analysis of noisy and high-dimensional functional data.Comment: 35 pages, 4 figure