126,032 research outputs found
Double-Edge Factor Graphs: Definition, Properties, and Examples
Some of the most interesting quantities associated with a factor graph are
its marginals and its partition sum. For factor graphs \emph{without cycles}
and moderate message update complexities, the sum-product algorithm (SPA) can
be used to efficiently compute these quantities exactly. Moreover, for various
classes of factor graphs \emph{with cycles}, the SPA has been successfully
applied to efficiently compute good approximations to these quantities. Note
that in the case of factor graphs with cycles, the local functions are usually
non-negative real-valued functions. In this paper we introduce a class of
factor graphs, called double-edge factor graphs (DE-FGs), which allow local
functions to be complex-valued and only require them, in some suitable sense,
to be positive semi-definite. We discuss various properties of the SPA when
running it on DE-FGs and we show promising numerical results for various
example DE-FGs, some of which have connections to quantum information
processing.Comment: Submitte
A Novel Stochastic Decoding of LDPC Codes with Quantitative Guarantees
Low-density parity-check codes, a class of capacity-approaching linear codes,
are particularly recognized for their efficient decoding scheme. The decoding
scheme, known as the sum-product, is an iterative algorithm consisting of
passing messages between variable and check nodes of the factor graph. The
sum-product algorithm is fully parallelizable, owing to the fact that all
messages can be update concurrently. However, since it requires extensive
number of highly interconnected wires, the fully-parallel implementation of the
sum-product on chips is exceedingly challenging. Stochastic decoding
algorithms, which exchange binary messages, are of great interest for
mitigating this challenge and have been the focus of extensive research over
the past decade. They significantly reduce the required wiring and
computational complexity of the message-passing algorithm. Even though
stochastic decoders have been shown extremely effective in practice, the
theoretical aspect and understanding of such algorithms remains limited at
large. Our main objective in this paper is to address this issue. We first
propose a novel algorithm referred to as the Markov based stochastic decoding.
Then, we provide concrete quantitative guarantees on its performance for
tree-structured as well as general factor graphs. More specifically, we provide
upper-bounds on the first and second moments of the error, illustrating that
the proposed algorithm is an asymptotically consistent estimate of the
sum-product algorithm. We also validate our theoretical predictions with
experimental results, showing we achieve comparable performance to other
practical stochastic decoders.Comment: This paper has been submitted to IEEE Transactions on Information
Theory on May 24th 201
Entropy Message Passing
The paper proposes a new message passing algorithm for cycle-free factor
graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as
sum-product message passing over the entropy semiring, which has previously
appeared in automata theory. The primary use of EMP is to compute the entropy
of a model. However, EMP can also be used to compute expressions that appear in
expectation maximization and in gradient descent algorithms.Comment: 5 pages, 1 figure, to appear in IEEE Transactions on Information
Theor
Local Message Passing on Frustrated Systems
Message passing on factor graphs is a powerful framework for probabilistic
inference, which finds important applications in various scientific domains.
The most wide-spread message passing scheme is the sum-product algorithm (SPA)
which gives exact results on trees but often fails on graphs with many small
cycles. We search for an alternative message passing algorithm that works
particularly well on such cyclic graphs. Therefore, we challenge the extrinsic
principle of the SPA, which loses its objective on graphs with cycles. We
further replace the local SPA message update rule at the factor nodes of the
underlying graph with a generic mapping, which is optimized in a data-driven
fashion. These modifications lead to a considerable improvement in performance
while preserving the simplicity of the SPA. We evaluate our method for two
classes of cyclic graphs: the 2x2 fully connected Ising grid and factor graphs
for symbol detection on linear communication channels with inter-symbol
interference. To enable the method for large graphs as they occur in practical
applications, we develop a novel loss function that is inspired by the Bethe
approximation from statistical physics and allows for training in an
unsupervised fashion.Comment: To appear at UAI 202
Structural Optimization of Factor Graphs for Symbol Detection via Continuous Clustering and Machine Learning
We propose a novel method to optimize the structure of factor graphs for
graph-based inference. As an example inference task, we consider symbol
detection on linear inter-symbol interference channels. The factor graph
framework has the potential to yield low-complexity symbol detectors. However,
the sum-product algorithm on cyclic factor graphs is suboptimal and its
performance is highly sensitive to the underlying graph. Therefore, we optimize
the structure of the underlying factor graphs in an end-to-end manner using
machine learning. For that purpose, we transform the structural optimization
into a clustering problem of low-degree factor nodes that incorporates the
known channel model into the optimization. Furthermore, we study the
combination of this approach with neural belief propagation, yielding
near-maximum a posteriori symbol detection performance for specific channels.Comment: Submitted to ICASSP 202
Sufficient conditions for convergence of the Sum-Product Algorithm
We derive novel conditions that guarantee convergence of the Sum-Product
algorithm (also known as Loopy Belief Propagation or simply Belief Propagation)
to a unique fixed point, irrespective of the initial messages. The
computational complexity of the conditions is polynomial in the number of
variables. In contrast with previously existing conditions, our results are
directly applicable to arbitrary factor graphs (with discrete variables) and
are shown to be valid also in the case of factors containing zeros, under some
additional conditions. We compare our bounds with existing ones, numerically
and, if possible, analytically. For binary variables with pairwise
interactions, we derive sufficient conditions that take into account local
evidence (i.e., single variable factors) and the type of pair interactions
(attractive or repulsive). It is shown empirically that this bound outperforms
existing bounds.Comment: 15 pages, 5 figures. Major changes and new results in this revised
version. Submitted to IEEE Transactions on Information Theor
Absorbing Set Analysis and Design of LDPC Codes from Transversal Designs over the AWGN Channel
In this paper we construct low-density parity-check (LDPC) codes from
transversal designs with low error-floors over the additive white Gaussian
noise (AWGN) channel. The constructed codes are based on transversal designs
that arise from sets of mutually orthogonal Latin squares (MOLS) with cyclic
structure. For lowering the error-floors, our approach is twofold: First, we
give an exhaustive classification of so-called absorbing sets that may occur in
the factor graphs of the given codes. These purely combinatorial substructures
are known to be the main cause of decoding errors in the error-floor region
over the AWGN channel by decoding with the standard sum-product algorithm
(SPA). Second, based on this classification, we exploit the specific structure
of the presented codes to eliminate the most harmful absorbing sets and derive
powerful constraints for the proper choice of code parameters in order to
obtain codes with an optimized error-floor performance.Comment: 15 pages. arXiv admin note: text overlap with arXiv:1306.511
Efficient Maximum Likelihood Estimation for Pedigree Data with the Sum-Product Algorithm
In this paper, we analyze data sets consisting of pedigrees where the response is the age at onset of colorectal cancer (CRC). The occurrence of familial clusters of CRC suggests the existence of a latent, inheritable risk factor. We aimed to compute the probability of a family possessing this risk factor, as well as the hazard rate increase for these risk factor carriers. Due to the inheritability of this risk factor, the estimation necessitates a costly marginalization of the likelihood.
We therefore developed an EM algorithm by applying factor graphs and the sum-product algorithm in the E-step, reducing the computational complexity from exponential to linear in the number of family members.
Our algorithm is as precise as a direct likelihood maximization in a simulation study and a real family study on CRC risk. For 250 simulated families of size 19 and 21, the runtime of our algorithm is faster by a factor of 4 and 29, respectively. On the largest family (23 members) in the real data, our algorithm is 6 times faster.
We introduce a flexible and runtime-efficient tool for statistical inference in biomedical event data that opens the door for advanced analyses of pedigree data
Efficient Maximum Likelihood Estimation for Pedigree Data with the Sum-Product Algorithm
OBJECTIVE We analyze data sets consisting of pedigrees with age at onset of colorectal cancer (CRC) as phenotype. The occurrence of familial clusters of CRC suggests the existence of a latent, inheritable risk factor. We aimed to compute the probability of a family possessing this risk factor as well as the hazard rate increase for these risk factor carriers. Due to the inheritability of this risk factor, the estimation necessitates a costly marginalization of the likelihood. METHODS We propose an improved EM algorithm by applying factor graphs and the sum-product algorithm in the E-step. This reduces the computational complexity from exponential to linear in the number of family members. RESULTS Our algorithm is as precise as a direct likelihood maximization in a simulation study and a real family study on CRC risk. For 250 simulated families of size 19 and 21, the runtime of our algorithm is faster by a factor of 4 and 29, respectively. On the largest family (23 members) in the real data, our algorithm is 6 times faster. CONCLUSION We introduce a flexible and runtime-efficient tool for statistical inference in biomedical event data with latent variables that opens the door for advanced analyses of pedigree data
- …