1,465 research outputs found
An Ensemble Framework for Detecting Community Changes in Dynamic Networks
Dynamic networks, especially those representing social networks, undergo
constant evolution of their community structure over time. Nodes can migrate
between different communities, communities can split into multiple new
communities, communities can merge together, etc. In order to represent dynamic
networks with evolving communities it is essential to use a dynamic model
rather than a static one. Here we use a dynamic stochastic block model where
the underlying block model is different at different times. In order to
represent the structural changes expressed by this dynamic model the network
will be split into discrete time segments and a clustering algorithm will
assign block memberships for each segment. In this paper we show that using an
ensemble of clustering assignments accommodates for the variance in scalable
clustering algorithms and produces superior results in terms of
pairwise-precision and pairwise-recall. We also demonstrate that the dynamic
clustering produced by the ensemble can be visualized as a flowchart which
encapsulates the community evolution succinctly.Comment: 6 pages, under submission to HPEC Graph Challeng
Beyond Hartigan Consistency: Merge Distortion Metric for Hierarchical Clustering
Hierarchical clustering is a popular method for analyzing data which
associates a tree to a dataset. Hartigan consistency has been used extensively
as a framework to analyze such clustering algorithms from a statistical point
of view. Still, as we show in the paper, a tree which is Hartigan consistent
with a given density can look very different than the correct limit tree.
Specifically, Hartigan consistency permits two types of undesirable
configurations which we term over-segmentation and improper nesting. Moreover,
Hartigan consistency is a limit property and does not directly quantify
difference between trees.
In this paper we identify two limit properties, separation and minimality,
which address both over-segmentation and improper nesting and together imply
(but are not implied by) Hartigan consistency. We proceed to introduce a merge
distortion metric between hierarchical clusterings and show that convergence in
our distance implies both separation and minimality. We also prove that uniform
separation and minimality imply convergence in the merge distortion metric.
Furthermore, we show that our merge distortion metric is stable under
perturbations of the density.
Finally, we demonstrate applicability of these concepts by proving
convergence results for two clustering algorithms. First, we show convergence
(and hence separation and minimality) of the recent robust single linkage
algorithm of Chaudhuri and Dasgupta (2010). Second, we provide convergence
results on manifolds for topological split tree clustering
Machine learning of hierarchical clustering to segment 2D and 3D images
We aim to improve segmentation through the use of machine learning tools
during region agglomeration. We propose an active learning approach for
performing hierarchical agglomerative segmentation from superpixels. Our method
combines multiple features at all scales of the agglomerative process, works
for data with an arbitrary number of dimensions, and scales to very large
datasets. We advocate the use of variation of information to measure
segmentation accuracy, particularly in 3D electron microscopy (EM) images of
neural tissue, and using this metric demonstrate an improvement over competing
algorithms in EM and natural images.Comment: 15 pages, 8 figure
- …