1,785 research outputs found
Community-aware network sparsification
Network sparsification aims to reduce the number of edges of a network while maintaining its structural properties; such properties include shortest paths, cuts, spectral measures, or network modularity. Sparsification has multiple applications, such as, speeding up graph-mining algorithms, graph visualization, as well as identifying the important network edges.
In this paper we consider a novel formulation of the network-sparsification problem. In addition to the network, we also consider as input a set of communities. The goal is to sparsify the network so as to preserve the network structure with respect to the given communities. We introduce two variants of the community-aware sparsification problem, leading to sparsifiers that satisfy different connectedness community properties. From the technical point of view, we prove hardness results and devise effective approximation algorithms. Our experimental results on a large collection of datasets demonstrate the effectiveness of our algorithms.https://epubs.siam.org/doi/10.1137/1.9781611974973.48Accepted manuscrip
Generating directed networks with prescribed Laplacian spectra
Complex real-world phenomena are often modeled as dynamical systems on
networks. In many cases of interest, the spectrum of the underlying graph
Laplacian sets the system stability and ultimately shapes the matter or
information flow. This motivates devising suitable strategies, with rigorous
mathematical foundation, to generate Laplacian that possess prescribed spectra.
In this paper, we show that a weighted Laplacians can be constructed so as to
exactly realize a desired complex spectrum. The method configures as a non
trivial generalization of existing recipes which assume the spectra to be real.
Applications of the proposed technique to (i) a network of Stuart-Landau
oscillators and (ii) to the Kuramoto model are discussed. Synchronization can
be enforced by assuming a properly engineered, signed and weighted, adjacency
matrix to rule the pattern of pairing interactions
The Convergence of Sparsified Gradient Methods
Distributed training of massive machine learning models, in particular deep
neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace.
Several families of communication-reduction methods, such as quantization,
large-batch methods, and gradient sparsification, have been proposed. To date,
gradient sparsification methods - where each node sorts gradients by magnitude,
and only communicates a subset of the components, accumulating the rest locally
- are known to yield some of the largest practical gains. Such methods can
reduce the amount of communication per step by up to three orders of magnitude,
while preserving model accuracy. Yet, this family of methods currently has no
theoretical justification.
This is the question we address in this paper. We prove that, under analytic
assumptions, sparsifying gradients by magnitude with local error correction
provides convergence guarantees, for both convex and non-convex smooth
objectives, for data-parallel SGD. The main insight is that sparsification
methods implicitly maintain bounds on the maximum impact of stale updates,
thanks to selection by magnitude. Our analysis and empirical validation also
reveal that these methods do require analytical conditions to converge well,
justifying existing heuristics.Comment: NIPS 2018 - Advances in Neural Information Processing Systems;
Authors in alphabetic orde
- …