Approximating the Permanent with Fractional Belief Propagation

We discuss schemes for exact and approximate computations of permanents, and compare them with each other. Specifically, we analyze the Belief Propagation (BP) approach and its Fractional Belief Propagation (FBP) generalization for computing the permanent of a non-negative matrix. Known bounds and conjectures are verified in experiments, and some new theoretical relations, bounds and conjectures are proposed. The Fractional Free Energy (FFE) functional is parameterized by a scalar parameter $\gamma\in[-1;1]$, where $\gamma=-1$ corresponds to the BP limit and $\gamma=1$ corresponds to the exclusion principle (but ignoring perfect matching constraints) Mean-Field (MF) limit. FFE shows monotonicity and continuity with respect to $\gamma$. For every non-negative matrix, we define its special value $\gamma_*\in[-1;0]$ to be the $\gamma$ for which the minimum of the $\gamma$-parameterized FFE functional is equal to the permanent of the matrix, where the lower and upper bounds of the $\gamma$-interval corresponds to respective bounds for the permanent. Our experimental analysis suggests that the distribution of $\gamma_*$ varies for different ensembles but $\gamma_*$ always lies within the $[-1;-1/2]$ interval. Moreover, for all ensembles considered the behavior of $\gamma_*$ is highly distinctive, offering an emprirical practical guidance for estimating permanents of non-negative matrices via the FFE approach.Comment: 42 pages, 14 figure

Palette-colouring: a belief-propagation approach

We consider a variation of the prototype combinatorial-optimisation problem known as graph-colouring. Our optimisation goal is to colour the vertices of a graph with a fixed number of colours, in a way to maximise the number of different colours present in the set of nearest neighbours of each given vertex. This problem, which we pictorially call "palette-colouring", has been recently addressed as a basic example of problem arising in the context of distributed data storage. Even though it has not been proved to be NP complete, random search algorithms find the problem hard to solve. Heuristics based on a naive belief propagation algorithm are observed to work quite well in certain conditions. In this paper, we build upon the mentioned result, working out the correct belief propagation algorithm, which needs to take into account the many-body nature of the constraints present in this problem. This method improves the naive belief propagation approach, at the cost of increased computational effort. We also investigate the emergence of a satisfiable to unsatisfiable "phase transition" as a function of the vertex mean degree, for different ensembles of sparse random graphs in the large size ("thermodynamic") limit.Comment: 22 pages, 7 figure

Analysis and Optimization of Aperture Design in Computational Imaging

There is growing interest in the use of coded aperture imaging systems for a variety of applications. Using an analysis framework based on mutual information, we examine the fundamental limits of such systems---and the associated optimum aperture coding---under simple but meaningful propagation and sensor models. Among other results, we show that when thermal noise dominates, spectrally-flat masks, which have 50% transmissivity, are optimal, but that when shot noise dominates, randomly generated masks with lower transmissivity offer greater performance. We also provide comparisons to classical pinhole cameras

Iterative Quantization Using Codes On Graphs

We study codes on graphs combined with an iterative message passing algorithm for quantization. Specifically, we consider the binary erasure quantization (BEQ) problem which is the dual of the binary erasure channel (BEC) coding problem. We show that duals of capacity achieving codes for the BEC yield codes which approach the minimum possible rate for the BEQ. In contrast, low density parity check codes cannot achieve the minimum rate unless their density grows at least logarithmically with block length. Furthermore, we show that duals of efficient iterative decoding algorithms for the BEC yield efficient encoding algorithms for the BEQ. Hence our results suggest that graphical models may yield near optimal codes in source coding as well as in channel coding and that duality plays a key role in such constructions.Comment: 10 page

Belief Propagation Min-Sum Algorithm for Generalized Min-Cost Network Flow

Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical guarantees. This paper extends the short list of special cases where correctness and/or convergence of a Belief Propagation algorithm is proven. We generalize formulation of Min-Sum Network Flow problem by relaxing the flow conservation (balance) constraints and then proving that the Belief Propagation algorithm converges to the exact result

Hierarchical and High-Girth QC LDPC Codes

We present a general approach to designing capacity-approaching high-girth low-density parity-check (LDPC) codes that are friendly to hardware implementation. Our methodology starts by defining a new class of "hierarchical" quasi-cyclic (HQC) LDPC codes that generalizes the structure of quasi-cyclic (QC) LDPC codes. Whereas the parity check matrices of QC LDPC codes are composed of circulant sub-matrices, those of HQC LDPC codes are composed of a hierarchy of circulant sub-matrices that are in turn constructed from circulant sub-matrices, and so on, through some number of levels. We show how to map any class of codes defined using a protograph into a family of HQC LDPC codes. Next, we present a girth-maximizing algorithm that optimizes the degrees of freedom within the family of codes to yield a high-girth HQC LDPC code. Finally, we discuss how certain characteristics of a code protograph will lead to inevitable short cycles, and show that these short cycles can be eliminated using a "squashing" procedure that results in a high-girth QC LDPC code, although not a hierarchical one. We illustrate our approach with designed examples of girth-10 QC LDPC codes obtained from protographs of one-sided spatially-coupled codes.Comment: Submitted to IEEE Transactions on Information THeor