13,773 research outputs found
Computing the Unique Information
Given a pair of predictor variables and a response variable, how much
information do the predictors have about the response, and how is this
information distributed between unique, redundant, and synergistic components?
Recent work has proposed to quantify the unique component of the decomposition
as the minimum value of the conditional mutual information over a constrained
set of information channels. We present an efficient iterative divergence
minimization algorithm to solve this optimization problem with convergence
guarantees and evaluate its performance against other techniques.Comment: To appear in 2018 IEEE International Symposium on Information Theory
(ISIT); 18 pages; 4 figures, 1 Table; Github link to source code:
https://github.com/infodeco/computeU
Accelerating MCMC via Parallel Predictive Prefetching
We present a general framework for accelerating a large class of widely used
Markov chain Monte Carlo (MCMC) algorithms. Our approach exploits fast,
iterative approximations to the target density to speculatively evaluate many
potential future steps of the chain in parallel. The approach can accelerate
computation of the target distribution of a Bayesian inference problem, without
compromising exactness, by exploiting subsets of data. It takes advantage of
whatever parallel resources are available, but produces results exactly
equivalent to standard serial execution. In the initial burn-in phase of chain
evaluation, it achieves speedup over serial evaluation that is close to linear
in the number of available cores
- …