219 research outputs found
Approximating Edit Distance Within Constant Factor in Truly Sub-Quadratic Time
Edit distance is a measure of similarity of two strings based on the minimum
number of character insertions, deletions, and substitutions required to
transform one string into the other. The edit distance can be computed exactly
using a dynamic programming algorithm that runs in quadratic time. Andoni,
Krauthgamer and Onak (2010) gave a nearly linear time algorithm that
approximates edit distance within approximation factor .
In this paper, we provide an algorithm with running time
that approximates the edit distance within a constant
factor
Training Overparametrized Neural Networks in Sublinear Time
The success of deep learning comes at a tremendous computational and energy
cost, and the scalability of training massively overparametrized neural
networks is becoming a real barrier to the progress of artificial intelligence
(AI). Despite the popularity and low cost-per-iteration of traditional
backpropagation via gradient decent, stochastic gradient descent (SGD) has
prohibitive convergence rate in non-convex settings, both in theory and
practice.
To mitigate this cost, recent works have proposed to employ alternative
(Newton-type) training methods with much faster convergence rate, albeit with
higher cost-per-iteration. For a typical neural network with
parameters and input batch of datapoints in
, the previous work of [Brand, Peng, Song, and Weinstein,
ITCS'2021] requires time per iteration. In this paper, we
present a novel training method that requires only
amortized time in the same overparametrized regime, where
is some fixed constant. This method relies on a new and alternative view of
neural networks, as a set of binary search trees, where each iteration
corresponds to modifying a small subset of the nodes in the tree. We believe
this view would have further applications in the design and analysis of deep
neural networks (DNNs)
Space- and Time-Efficient Algorithm for Maintaining Dense Subgraphs on One-Pass Dynamic Streams
While in many graph mining applications it is crucial to handle a stream of
updates efficiently in terms of {\em both} time and space, not much was known
about achieving such type of algorithm. In this paper we study this issue for a
problem which lies at the core of many graph mining applications called {\em
densest subgraph problem}. We develop an algorithm that achieves time- and
space-efficiency for this problem simultaneously. It is one of the first of its
kind for graph problems to the best of our knowledge.
In a graph , the "density" of a subgraph induced by a subset of
nodes is defined as , where is the set of
edges in with both endpoints in . In the densest subgraph problem, the
goal is to find a subset of nodes that maximizes the density of the
corresponding induced subgraph. For any , we present a dynamic
algorithm that, with high probability, maintains a -approximation
to the densest subgraph problem under a sequence of edge insertions and
deletions in a graph with nodes. It uses space, and has an
amortized update time of and a query time of . Here,
hides a O(\poly\log_{1+\epsilon} n) term. The approximation ratio
can be improved to at the cost of increasing the query time to
. It can be extended to a -approximation
sublinear-time algorithm and a distributed-streaming algorithm. Our algorithm
is the first streaming algorithm that can maintain the densest subgraph in {\em
one pass}. The previously best algorithm in this setting required
passes [Bahmani, Kumar and Vassilvitskii, VLDB'12]. The space required by our
algorithm is tight up to a polylogarithmic factor.Comment: A preliminary version of this paper appeared in STOC 201
Algorithmic linear dimension reduction in the l_1 norm for sparse vectors
This paper develops a new method for recovering m-sparse signals that is
simultaneously uniform and quick. We present a reconstruction algorithm whose
run time, O(m log^2(m) log^2(d)), is sublinear in the length d of the signal.
The reconstruction error is within a logarithmic factor (in m) of the optimal
m-term approximation error in l_1. In particular, the algorithm recovers
m-sparse signals perfectly and noisy signals are recovered with polylogarithmic
distortion. Our algorithm makes O(m log^2 (d)) measurements, which is within a
logarithmic factor of optimal. We also present a small-space implementation of
the algorithm. These sketching techniques and the corresponding reconstruction
algorithms provide an algorithmic dimension reduction in the l_1 norm. In
particular, vectors of support m in dimension d can be linearly embedded into
O(m log^2 d) dimensions with polylogarithmic distortion. We can reconstruct a
vector from its low-dimensional sketch in time O(m log^2(m) log^2(d)).
Furthermore, this reconstruction is stable and robust under small
perturbations
- …