1,474 research outputs found
Towards Building Deep Networks with Bayesian Factor Graphs
We propose a Multi-Layer Network based on the Bayesian framework of the
Factor Graphs in Reduced Normal Form (FGrn) applied to a two-dimensional
lattice. The Latent Variable Model (LVM) is the basic building block of a
quadtree hierarchy built on top of a bottom layer of random variables that
represent pixels of an image, a feature map, or more generally a collection of
spatially distributed discrete variables. The multi-layer architecture
implements a hierarchical data representation that, via belief propagation, can
be used for learning and inference. Typical uses are pattern completion,
correction and classification. The FGrn paradigm provides great flexibility and
modularity and appears as a promising candidate for building deep networks: the
system can be easily extended by introducing new and different (in cardinality
and in type) variables. Prior knowledge, or supervised information, can be
introduced at different scales. The FGrn paradigm provides a handy way for
building all kinds of architectures by interconnecting only three types of
units: Single Input Single Output (SISO) blocks, Sources and Replicators. The
network is designed like a circuit diagram and the belief messages flow
bidirectionally in the whole system. The learning algorithms operate only
locally within each block. The framework is demonstrated in this paper in a
three-layer structure applied to images extracted from a standard data set.Comment: Submitted for journal publicatio
Model selection and hypothesis testing for large-scale network models with overlapping groups
The effort to understand network systems in increasing detail has resulted in
a diversity of methods designed to extract their large-scale structure from
data. Unfortunately, many of these methods yield diverging descriptions of the
same network, making both the comparison and understanding of their results a
difficult challenge. A possible solution to this outstanding issue is to shift
the focus away from ad hoc methods and move towards more principled approaches
based on statistical inference of generative models. As a result, we face
instead the more well-defined task of selecting between competing generative
processes, which can be done under a unified probabilistic framework. Here, we
consider the comparison between a variety of generative models including
features such as degree correction, where nodes with arbitrary degrees can
belong to the same group, and community overlap, where nodes are allowed to
belong to more than one group. Because such model variants possess an
increasing number of parameters, they become prone to overfitting. In this
work, we present a method of model selection based on the minimum description
length criterion and posterior odds ratios that is capable of fully accounting
for the increased degrees of freedom of the larger models, and selects the best
one according to the statistical evidence available in the data. In applying
this method to many empirical unweighted networks from different fields, we
observe that community overlap is very often not supported by statistical
evidence and is selected as a better model only for a minority of them. On the
other hand, we find that degree correction tends to be almost universally
favored by the available data, implying that intrinsic node proprieties (as
opposed to group properties) are often an essential ingredient of network
formation.Comment: 20 pages,7 figures, 1 tabl
- …