Bayesian Inference of Log Determinants

The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few thousand. In the spirit of probabilistic numerics, we reinterpret the problem of computing the log-determinant as a Bayesian inference problem. In particular, we combine prior knowledge in the form of bounds from matrix theory and evidence derived from stochastic trace estimation to obtain probabilistic estimates for the log-determinant and its associated uncertainty within a given computational budget. Beyond its novelty and theoretic appeal, the performance of our proposal is competitive with state-of-the-art approaches to approximating the log-determinant, while also quantifying the uncertainty due to budget-constrained evidence.Comment: 12 pages, 3 figure

Suboptimal subspace construction for log-determinant approximation

Variance reduction is a crucial idea for Monte Carlo simulation and the stochastic Lanczos quadrature method is a dedicated method to approximate the trace of a matrix function. Inspired by their advantages, we combine these two techniques to approximate the log-determinant of large-scale symmetric positive definite matrices. Key questions to be answered for such a method are how to construct or choose an appropriate projection subspace and derive guaranteed theoretical analysis. This paper applies some probabilistic approaches including the projection-cost-preserving sketch and matrix concentration inequalities to construct a suboptimal subspace. Furthermore, we provide some insights on choosing design parameters in the underlying algorithm by deriving corresponding approximation error and probabilistic error estimations. Numerical experiments demonstrate our method's effectiveness and illustrate the quality of the derived error bounds

On a symbolic representation of non-central Wishart random matrices with applications

By using a symbolic method, known in the literature as the classical umbral calculus, the trace of a non-central Wishart random matrix is represented as the convolution of the trace of its central component and of a formal variable involving traces of its non-centrality matrix. Thanks to this representation, the moments of this random matrix are proved to be a Sheffer polynomial sequence, allowing us to recover several properties. The multivariate symbolic method generalizes the employment of Sheffer representation and a closed form formula for computing joint moments and cumulants (also normalized) is given. By using this closed form formula and a combinatorial device, known in the literature as necklace, an efficient algorithm for their computations is set up. Applications are given to the computation of permanents as well as to the characterization of inherited estimators of cumulants, which turn useful in dealing with minors of non-central Wishart random matrices. An asymptotic approximation of generalized moments involving free probability is proposed.Comment: Journal of Multivariate Analysis (2014