Information Recovery from Pairwise Measurements

A variety of information processing tasks in practice involve recovering $n$ objects from single-shot graph-based measurements, particularly those taken over the edges of some measurement graph $\mathcal{G}$. This paper concerns the situation where each object takes value over a group of $M$ different values, and where one is interested to recover all these values based on observations of certain pairwise relations over $\mathcal{G}$. The imperfection of measurements presents two major challenges for information recovery: 1) $\textit{inaccuracy}$: a (dominant) portion $1-p$ of measurements are corrupted; 2) $\textit{incompleteness}$: a significant fraction of pairs are unobservable, i.e. $\mathcal{G}$ can be highly sparse. Under a natural random outlier model, we characterize the $\textit{minimax recovery rate}$, that is, the critical threshold of non-corruption rate $p$ below which exact information recovery is infeasible. This accommodates a very general class of pairwise relations. For various homogeneous random graph models (e.g. Erdos Renyi random graphs, random geometric graphs, small world graphs), the minimax recovery rate depends almost exclusively on the edge sparsity of the measurement graph $\mathcal{G}$ irrespective of other graphical metrics. This fundamental limit decays with the group size $M$ at a square root rate before entering a connectivity-limited regime. Under the Erdos Renyi random graph, a tractable combinatorial algorithm is proposed to approach the limit for large $M$ ($M=n^{\Omega(1)}$), while order-optimal recovery is enabled by semidefinite programs in the small $M$ regime. The extended (and most updated) version of this work can be found at (http://arxiv.org/abs/1504.01369).Comment: This version is no longer updated -- please find the latest version at (arXiv:1504.01369

Dynamics and Performance of Susceptibility Propagation on Synthetic Data

We study the performance and convergence properties of the Susceptibility Propagation (SusP) algorithm for solving the Inverse Ising problem. We first study how the temperature parameter (T) in a Sherrington-Kirkpatrick model generating the data influences the performance and convergence of the algorithm. We find that at the high temperature regime (T>4), the algorithm performs well and its quality is only limited by the quality of the supplied data. In the low temperature regime (T<4), we find that the algorithm typically does not converge, yielding diverging values for the couplings. However, we show that by stopping the algorithm at the right time before divergence becomes serious, good reconstruction can be achieved down to T~2. We then show that dense connectivity, loopiness of the connectivity, and high absolute magnetization all have deteriorating effects on the performance of the algorithm. When absolute magnetization is high, we show that other methods can be work better than SusP. Finally, we show that for neural data with high absolute magnetization, SusP performs less well than TAP inversion.Comment: 9 pages, 7 figure

Gossip Algorithms for Distributed Signal Processing

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This article presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.Comment: Submitted to Proceedings of the IEEE, 29 page

Convergence analysis of the information matrix in Gaussian belief propagation

Gaussian belief propagation (BP) has been widely used for distributed estimation in large-scale networks such as the smart grid, communication networks, and social networks, where local measurements/observations are scattered over a wide geographical area. However, the convergence of Gaus- sian BP is still an open issue. In this paper, we consider the convergence of Gaussian BP, focusing in particular on the convergence of the information matrix. We show analytically that the exchanged message information matrix converges for arbitrary positive semidefinite initial value, and its dis- tance to the unique positive definite limit matrix decreases exponentially fast.Comment: arXiv admin note: substantial text overlap with arXiv:1611.0201