61,636 research outputs found
Recovering Structured Probability Matrices
We consider the problem of accurately recovering a matrix B of size M by M ,
which represents a probability distribution over M2 outcomes, given access to
an observed matrix of "counts" generated by taking independent samples from the
distribution B. How can structural properties of the underlying matrix B be
leveraged to yield computationally efficient and information theoretically
optimal reconstruction algorithms? When can accurate reconstruction be
accomplished in the sparse data regime? This basic problem lies at the core of
a number of questions that are currently being considered by different
communities, including building recommendation systems and collaborative
filtering in the sparse data regime, community detection in sparse random
graphs, learning structured models such as topic models or hidden Markov
models, and the efforts from the natural language processing community to
compute "word embeddings".
Our results apply to the setting where B has a low rank structure. For this
setting, we propose an efficient algorithm that accurately recovers the
underlying M by M matrix using Theta(M) samples. This result easily translates
to Theta(M) sample algorithms for learning topic models and learning hidden
Markov Models. These linear sample complexities are optimal, up to constant
factors, in an extremely strong sense: even testing basic properties of the
underlying matrix (such as whether it has rank 1 or 2) requires Omega(M)
samples. We provide an even stronger lower bound where distinguishing whether a
sequence of observations were drawn from the uniform distribution over M
observations versus being generated by an HMM with two hidden states requires
Omega(M) observations. This precludes sublinear-sample hypothesis tests for
basic properties, such as identity or uniformity, as well as sublinear sample
estimators for quantities such as the entropy rate of HMMs
A semidefinite program for unbalanced multisection in the stochastic block model
We propose a semidefinite programming (SDP) algorithm for community detection
in the stochastic block model, a popular model for networks with latent
community structure. We prove that our algorithm achieves exact recovery of the
latent communities, up to the information-theoretic limits determined by Abbe
and Sandon (2015). Our result extends prior SDP approaches by allowing for many
communities of different sizes. By virtue of a semidefinite approach, our
algorithms succeed against a semirandom variant of the stochastic block model,
guaranteeing a form of robustness and generalization. We further explore how
semirandom models can lend insight into both the strengths and limitations of
SDPs in this setting.Comment: 29 page
Community detection and stochastic block models: recent developments
The stochastic block model (SBM) is a random graph model with planted
clusters. It is widely employed as a canonical model to study clustering and
community detection, and provides generally a fertile ground to study the
statistical and computational tradeoffs that arise in network and data
sciences.
This note surveys the recent developments that establish the fundamental
limits for community detection in the SBM, both with respect to
information-theoretic and computational thresholds, and for various recovery
requirements such as exact, partial and weak recovery (a.k.a., detection). The
main results discussed are the phase transitions for exact recovery at the
Chernoff-Hellinger threshold, the phase transition for weak recovery at the
Kesten-Stigum threshold, the optimal distortion-SNR tradeoff for partial
recovery, the learning of the SBM parameters and the gap between
information-theoretic and computational thresholds.
The note also covers some of the algorithms developed in the quest of
achieving the limits, in particular two-round algorithms via graph-splitting,
semi-definite programming, linearized belief propagation, classical and
nonbacktracking spectral methods. A few open problems are also discussed
Fundamental Limits on Data Acquisition: Trade-offs between Sample Complexity and Query Difficulty
We consider query-based data acquisition and the corresponding information
recovery problem, where the goal is to recover binary variables
(information bits) from parity measurements of those variables. The queries and
the corresponding parity measurements are designed using the encoding rule of
Fountain codes. By using Fountain codes, we can design potentially limitless
number of queries, and corresponding parity measurements, and guarantee that
the original information bits can be recovered with high probability from
any sufficiently large set of measurements of size . In the query design,
the average number of information bits that is associated with one parity
measurement is called query difficulty () and the minimum number of
measurements required to recover the information bits for a fixed
is called sample complexity (). We analyze the fundamental trade-offs
between the query difficulty and the sample complexity, and show that the
sample complexity of for some constant
is necessary and sufficient to recover information bits with high
probability as
- …