Distributed Convergence Verification for Gaussian Belief Propagation

Gaussian belief propagation (BP) is a computationally efficient method to approximate the marginal distribution and has been widely used for inference with high dimensional data as well as distributed estimation in large-scale networks. However, the convergence of Gaussian BP is still an open issue. Though sufficient convergence conditions have been studied in the literature, verifying these conditions requires gathering all the information over the whole network, which defeats the main advantage of distributed computing by using Gaussian BP. In this paper, we propose a novel sufficient convergence condition for Gaussian BP that applies to both the pairwise linear Gaussian model and to Gaussian Markov random fields. We show analytically that this sufficient convergence condition can be easily verified in a distributed way that satisfies the network topology constraint.Comment: accepted by Asilomar Conference on Signals, Systems, and Computers, 2017, Asilomar, Pacific Grove, CA. arXiv admin note: text overlap with arXiv:1706.0407

On Convergence of Approximate Message Passing

Approximate message passing is an iterative algorithm for compressed sensing and related applications. A solid theory about the performance and convergence of the algorithm exists for measurement matrices having iid entries of zero mean. However, it was observed by several authors that for more general matrices the algorithm often encounters convergence problems. In this paper we identify the reason of the non-convergence for measurement matrices with iid entries and non-zero mean in the context of Bayes optimal inference. Finally we demonstrate numerically that when the iterative update is changed from parallel to sequential the convergence is restored.Comment: 5 pages, 3 figure

Factored expectation propagation for input-output FHMM models in systems biology

We consider the problem of joint modelling of metabolic signals and gene expression in systems biology applications. We propose an approach based on input-output factorial hidden Markov models and propose a structured variational inference approach to infer the structure and states of the model. We start from the classical free form structured variational mean field approach and use a expectation propagation to approximate the expectations needed in the variational loop. We show that this corresponds to a factored expectation constrained approximate inference. We validate our model through extensive simulations and demonstrate its applicability on a real world bacterial data set

Determining the convergence of variance in Gaussian belief propagation via semi-definite programming

In order to compute the marginal distribution from a high dimensional distribution with loopy Gaussian belief propagation (BP), it is important to determine whether Gaussian BP would converge. In general, the convergence condition for Gaussian BP variance and mean are not necessarily the same, and this paper focuses on the convergence condition of Gaussian BP variance. In particular, by describing the message-passing process of Gaussian BP as a set of updating functions, the necessary and sufficient convergence condition of Gaussian BP variance is derived, with the converged variance proved to be independent of the initialization as long as it is greater or equal to zero. It is further proved that the convergence condition can be verified efficiently by solving a semi-definite programming (SDP) optimization problem. Numerical examples are presented to corroborate the established theories.published_or_final_versio

Convergence Analysis of the Variance in Gaussian Belief Propagation

It is known that Gaussian belief propagation (BP) is a low-complexity algorithm for (approximately) computing the marginal distribution of a high dimensional Gaussian distribu- tion. However, in loopy factor graph, it is important to determine whether Gaussian BP converges. In general, the convergence conditions for Gaussian BP variances and means are not nec- essarily the same, and this paper focuses on the convergence condition of Gaussian BP variances. In particular, by describing the message-passing process of Gaussian BP as a set of updating functions, the necessary and sufficient convergence conditions of Gaussian BP variances are derived under both synchronous and asynchronous schedulings, with the converged variances proved to be independent of the initialization as long as it is chosen from the proposed set. The necessary and sufficient convergence condition is further expressed in the form of a semi-definite programming (SDP) optimization problem, thus can be verified more efficiently compared to the existing convergence condition based on compu- tation tree. The relationship between the proposed convergence condition and the existing one based on computation tree is also established analytically. Numerical examples are presented to corroborate the established theories.published_or_final_versio