1,969 research outputs found
A Tensor Approach to Learning Mixed Membership Community Models
Community detection is the task of detecting hidden communities from observed
interactions. Guaranteed community detection has so far been mostly limited to
models with non-overlapping communities such as the stochastic block model. In
this paper, we remove this restriction, and provide guaranteed community
detection for a family of probabilistic network models with overlapping
communities, termed as the mixed membership Dirichlet model, first introduced
by Airoldi et al. This model allows for nodes to have fractional memberships in
multiple communities and assumes that the community memberships are drawn from
a Dirichlet distribution. Moreover, it contains the stochastic block model as a
special case. We propose a unified approach to learning these models via a
tensor spectral decomposition method. Our estimator is based on low-order
moment tensor of the observed network, consisting of 3-star counts. Our
learning method is fast and is based on simple linear algebraic operations,
e.g. singular value decomposition and tensor power iterations. We provide
guaranteed recovery of community memberships and model parameters and present a
careful finite sample analysis of our learning method. As an important special
case, our results match the best known scaling requirements for the
(homogeneous) stochastic block model
Computing a Nonnegative Matrix Factorization -- Provably
In the Nonnegative Matrix Factorization (NMF) problem we are given an nonnegative matrix and an integer . Our goal is to express
as where and are nonnegative matrices of size
and respectively. In some applications, it makes sense to ask
instead for the product to approximate -- i.e. (approximately)
minimize \norm{M - AW}_F where \norm{}_F denotes the Frobenius norm; we
refer to this as Approximate NMF. This problem has a rich history spanning
quantum mechanics, probability theory, data analysis, polyhedral combinatorics,
communication complexity, demography, chemometrics, etc. In the past decade NMF
has become enormously popular in machine learning, where and are
computed using a variety of local search heuristics. Vavasis proved that this
problem is NP-complete. We initiate a study of when this problem is solvable in
polynomial time:
1. We give a polynomial-time algorithm for exact and approximate NMF for
every constant . Indeed NMF is most interesting in applications precisely
when is small.
2. We complement this with a hardness result, that if exact NMF can be solved
in time , 3-SAT has a sub-exponential time algorithm. This rules
out substantial improvements to the above algorithm.
3. We give an algorithm that runs in time polynomial in , and
under the separablity condition identified by Donoho and Stodden in 2003. The
algorithm may be practical since it is simple and noise tolerant (under benign
assumptions). Separability is believed to hold in many practical settings.
To the best of our knowledge, this last result is the first example of a
polynomial-time algorithm that provably works under a non-trivial condition on
the input and we believe that this will be an interesting and important
direction for future work.Comment: 29 pages, 3 figure
Algorithmic and Statistical Perspectives on Large-Scale Data Analysis
In recent years, ideas from statistics and scientific computing have begun to
interact in increasingly sophisticated and fruitful ways with ideas from
computer science and the theory of algorithms to aid in the development of
improved worst-case algorithms that are useful for large-scale scientific and
Internet data analysis problems. In this chapter, I will describe two recent
examples---one having to do with selecting good columns or features from a (DNA
Single Nucleotide Polymorphism) data matrix, and the other having to do with
selecting good clusters or communities from a data graph (representing a social
or information network)---that drew on ideas from both areas and that may serve
as a model for exploiting complementary algorithmic and statistical
perspectives in order to solve applied large-scale data analysis problems.Comment: 33 pages. To appear in Uwe Naumann and Olaf Schenk, editors,
"Combinatorial Scientific Computing," Chapman and Hall/CRC Press, 201
Practical Attacks Against Graph-based Clustering
Graph modeling allows numerous security problems to be tackled in a general
way, however, little work has been done to understand their ability to
withstand adversarial attacks. We design and evaluate two novel graph attacks
against a state-of-the-art network-level, graph-based detection system. Our
work highlights areas in adversarial machine learning that have not yet been
addressed, specifically: graph-based clustering techniques, and a global
feature space where realistic attackers without perfect knowledge must be
accounted for (by the defenders) in order to be practical. Even though less
informed attackers can evade graph clustering with low cost, we show that some
practical defenses are possible.Comment: ACM CCS 201
Spectral Thresholds in the Bipartite Stochastic Block Model
We consider a bipartite stochastic block model on vertex sets and
, with planted partitions in each, and ask at what densities efficient
algorithms can recover the partition of the smaller vertex set.
When , multiple thresholds emerge. We first locate a sharp
threshold for detection of the partition, in the sense of the results of
\cite{mossel2012stochastic,mossel2013proof} and \cite{massoulie2014community}
for the stochastic block model. We then show that at a higher edge density, the
singular vectors of the rectangular biadjacency matrix exhibit a localization /
delocalization phase transition, giving recovery above the threshold and no
recovery below. Nevertheless, we propose a simple spectral algorithm, Diagonal
Deletion SVD, which recovers the partition at a nearly optimal edge density.
The bipartite stochastic block model studied here was used by
\cite{feldman2014algorithm} to give a unified algorithm for recovering planted
partitions and assignments in random hypergraphs and random -SAT formulae
respectively. Our results give the best known bounds for the clause density at
which solutions can be found efficiently in these models as well as showing a
barrier to further improvement via this reduction to the bipartite block model.Comment: updated version, will appear in COLT 201
Detecting Hidden Communities by Power Iterations with Connections to Vanilla Spectral Algorithms
Community detection in the stochastic block model is one of the central
problems of graph clustering. Since its introduction, many subsequent papers
have made great strides in solving and understanding this model. In this setup,
spectral algorithms have been one of the most widely used frameworks. However,
despite the long history of study, there are still unsolved challenges. One of
the main open problems is the design and analysis of "simple"(vanilla) spectral
algorithms, especially when the number of communities is large.
In this paper, we provide two algorithms. The first one is based on the
power-iteration method. It is a simple algorithm which only compares the rows
of the powered adjacency matrix. Our algorithm performs optimally (up to
logarithmic factors) compared to the best known bounds in the dense graph
regime by Van Vu (Combinatorics Probability and Computing, 2018). Furthermore,
our algorithm is also robust to the "small cluster barrier", recovering large
clusters in the presence of an arbitrary number of small clusters. Then based
on a connection between the powered adjacency matrix and eigenvectors, we
provide a vanilla spectral algorithm for large number of communities in the
balanced case. This answers an open question by Van Vu (Combinatorics
Probability and Computing, 2018) in the balanced case. Our methods also
partially solve technical barriers discussed by Abbe, Fan, Wang and Zhong
(Annals of Statistics, 2020).
In the technical side, we introduce a random partition method to analyze each
entry of a powered random matrix. This method can be viewed as an eigenvector
version of Wigner's trace method. Recall that Wigner's trace method links the
trace of powered matrix to eigenvalues. Our method links the whole powered
matrix to the span of eigenvectors. We expect our method to have more
applications in random matrix theory
- …