15,342 research outputs found
A Generic Approach for Escaping Saddle points
A central challenge to using first-order methods for optimizing nonconvex
problems is the presence of saddle points. First-order methods often get stuck
at saddle points, greatly deteriorating their performance. Typically, to escape
from saddles one has to use second-order methods. However, most works on
second-order methods rely extensively on expensive Hessian-based computations,
making them impractical in large-scale settings. To tackle this challenge, we
introduce a generic framework that minimizes Hessian based computations while
at the same time provably converging to second-order critical points. Our
framework carefully alternates between a first-order and a second-order
subroutine, using the latter only close to saddle points, and yields
convergence results competitive to the state-of-the-art. Empirical results
suggest that our strategy also enjoys a good practical performance
Stochastic Control via Entropy Compression
We consider an agent trying to bring a system to an acceptable state by
repeated probabilistic action. Several recent works on algorithmizations of the
Lovasz Local Lemma (LLL) can be seen as establishing sufficient conditions for
the agent to succeed. Here we study whether such stochastic control is also
possible in a noisy environment, where both the process of state-observation
and the process of state-evolution are subject to adversarial perturbation
(noise). The introduction of noise causes the tools developed for LLL
algorithmization to break down since the key LLL ingredient, the sparsity of
the causality (dependence) relationship, no longer holds. To overcome this
challenge we develop a new analysis where entropy plays a central role, both to
measure the rate at which progress towards an acceptable state is made and the
rate at which noise undoes this progress. The end result is a sufficient
condition that allows a smooth tradeoff between the intensity of the noise and
the amenability of the system, recovering an asymmetric LLL condition in the
noiseless case.Comment: 18 page
Community Detection on Evolving Graphs
Clustering is a fundamental step in many information-retrieval and data-mining applications. Detecting clusters in graphs is also a key tool for finding the community structure in social and behavioral networks. In many of these applications, the input graph evolves over time in a continual and decentralized manner, and, to maintain a good clustering, the clustering algorithm needs to repeatedly probe the graph. Furthermore, there are often limitations on the frequency of such probes, either imposed explicitly by the online platform (e.g., in the case of crawling proprietary social networks like twitter) or implicitly because of resource limitations (e.g., in the case of crawling the web). In this paper, we study a model of clustering on evolving graphs that captures this aspect of the problem. Our model is based on the classical stochastic block model, which has been used to assess rigorously the quality of various static clustering methods. In our model, the algorithm is supposed to reconstruct the planted clustering, given the ability to query for small pieces of local information about the graph, at a limited rate. We design and analyze clustering algorithms that work in this model, and show asymptotically tight upper and lower bounds on their accuracy. Finally, we perform simulations, which demonstrate that our main asymptotic results hold true also in practice
- …