8,589 research outputs found
Asymptotic Mutual Information for the Two-Groups Stochastic Block Model
We develop an information-theoretic view of the stochastic block model, a
popular statistical model for the large-scale structure of complex networks. A
graph from such a model is generated by first assigning vertex labels at
random from a finite alphabet, and then connecting vertices with edge
probabilities depending on the labels of the endpoints. In the case of the
symmetric two-group model, we establish an explicit `single-letter'
characterization of the per-vertex mutual information between the vertex labels
and the graph.
The explicit expression of the mutual information is intimately related to
estimation-theoretic quantities, and --in particular-- reveals a phase
transition at the critical point for community detection. Below the critical
point the per-vertex mutual information is asymptotically the same as if edges
were independent. Correspondingly, no algorithm can estimate the partition
better than random guessing. Conversely, above the threshold, the per-vertex
mutual information is strictly smaller than the independent-edges upper bound.
In this regime there exists a procedure that estimates the vertex labels better
than random guessing.Comment: 41 pages, 3 pdf figure
Stochastic blockmodels and community structure in networks
Stochastic blockmodels have been proposed as a tool for detecting community
structure in networks as well as for generating synthetic networks for use as
benchmarks. Most blockmodels, however, ignore variation in vertex degree,
making them unsuitable for applications to real-world networks, which typically
display broad degree distributions that can significantly distort the results.
Here we demonstrate how the generalization of blockmodels to incorporate this
missing element leads to an improved objective function for community detection
in complex networks. We also propose a heuristic algorithm for community
detection using this objective function or its non-degree-corrected counterpart
and show that the degree-corrected version dramatically outperforms the
uncorrected one in both real-world and synthetic networks.Comment: 11 pages, 3 figure
Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
Factorizing low-rank matrices has many applications in machine learning and
statistics. For probabilistic models in the Bayes optimal setting, a general
expression for the mutual information has been proposed using heuristic
statistical physics computations, and proven in few specific cases. Here, we
show how to rigorously prove the conjectured formula for the symmetric rank-one
case. This allows to express the minimal mean-square-error and to characterize
the detectability phase transitions in a large set of estimation problems
ranging from community detection to sparse PCA. We also show that for a large
set of parameters, an iterative algorithm called approximate message-passing is
Bayes optimal. There exists, however, a gap between what currently known
polynomial algorithms can do and what is expected information theoretically.
Additionally, the proof technique has an interest of its own and exploits three
essential ingredients: the interpolation method introduced in statistical
physics by Guerra, the analysis of the approximate message-passing algorithm
and the theory of spatial coupling and threshold saturation in coding. Our
approach is generic and applicable to other open problems in statistical
estimation where heuristic statistical physics predictions are available
Reduction of Markov Chains using a Value-of-Information-Based Approach
In this paper, we propose an approach to obtain reduced-order models of
Markov chains. Our approach is composed of two information-theoretic processes.
The first is a means of comparing pairs of stationary chains on different state
spaces, which is done via the negative Kullback-Leibler divergence defined on a
model joint space. Model reduction is achieved by solving a
value-of-information criterion with respect to this divergence. Optimizing the
criterion leads to a probabilistic partitioning of the states in the high-order
Markov chain. A single free parameter that emerges through the optimization
process dictates both the partition uncertainty and the number of state groups.
We provide a data-driven means of choosing the `optimal' value of this free
parameter, which sidesteps needing to a priori know the number of state groups
in an arbitrary chain.Comment: Submitted to Entrop
Community detection and stochastic block models: recent developments
The stochastic block model (SBM) is a random graph model with planted
clusters. It is widely employed as a canonical model to study clustering and
community detection, and provides generally a fertile ground to study the
statistical and computational tradeoffs that arise in network and data
sciences.
This note surveys the recent developments that establish the fundamental
limits for community detection in the SBM, both with respect to
information-theoretic and computational thresholds, and for various recovery
requirements such as exact, partial and weak recovery (a.k.a., detection). The
main results discussed are the phase transitions for exact recovery at the
Chernoff-Hellinger threshold, the phase transition for weak recovery at the
Kesten-Stigum threshold, the optimal distortion-SNR tradeoff for partial
recovery, the learning of the SBM parameters and the gap between
information-theoretic and computational thresholds.
The note also covers some of the algorithms developed in the quest of
achieving the limits, in particular two-round algorithms via graph-splitting,
semi-definite programming, linearized belief propagation, classical and
nonbacktracking spectral methods. A few open problems are also discussed
Finding communities in sparse networks
Spectral algorithms based on matrix representations of networks are often
used to detect communities but classic spectral methods based on the adjacency
matrix and its variants fail to detect communities in sparse networks. New
spectral methods based on non-backtracking random walks have recently been
introduced that successfully detect communities in many sparse networks.
However, the spectrum of non-backtracking random walks ignores hanging trees in
networks that can contain information about the community structure of
networks. We introduce the reluctant backtracking operators that explicitly
account for hanging trees as they admit a small probability of returning to the
immediately previous node unlike the non-backtracking operators that forbid an
immediate return. We show that the reluctant backtracking operators can detect
communities in certain sparse networks where the non-backtracking operators
cannot while performing comparably on benchmark stochastic block model networks
and real world networks. We also show that the spectrum of the reluctant
backtracking operator approximately optimises the standard modularity function
similar to the flow matrix. Interestingly, for this family of non- and
reluctant-backtracking operators the main determinant of performance on
real-world networks is whether or not they are normalised to conserve
probability at each node.Comment: 11 pages, 4 figure
- …