12 research outputs found
Deterministic Sparse Fourier Transform with an ?_{?} Guarantee
In this paper we revisit the deterministic version of the Sparse Fourier
Transform problem, which asks to read only a few entries of and design a recovery algorithm such that the output of the
algorithm approximates , the Discrete Fourier Transform (DFT) of .
The randomized case has been well-understood, while the main work in the
deterministic case is that of Merhi et al.\@ (J Fourier Anal Appl 2018), which
obtains samples and a similar runtime
with the guarantee. We focus on the stronger
guarantee and the closely related problem of incoherent
matrices. We list our contributions as follows.
1. We find a deterministic collection of samples for the
recovery in time , and a deterministic
collection of samples for the sparse
recovery in time .
2. We give new deterministic constructions of incoherent matrices that are
row-sampled submatrices of the DFT matrix, via a derandomization of Bernstein's
inequality and bounds on exponential sums considered in analytic number theory.
Our first construction matches a previous randomized construction of Nelson,
Nguyen and Woodruff (RANDOM'12), where there was no constraint on the form of
the incoherent matrix.
Our algorithms are nearly sample-optimal, since a lower bound of is known, even for the case where the sensing matrix can be
arbitrarily designed. A similar lower bound of is
known for incoherent matrices.Comment: ICALP 2020--presentation improved according to reviewers' comment
Federated Empirical Risk Minimization via Second-Order Method
Many convex optimization problems with important applications in machine
learning are formulated as empirical risk minimization (ERM). There are several
examples: linear and logistic regression, LASSO, kernel regression, quantile
regression, -norm regression, support vector machines (SVM), and mean-field
variational inference. To improve data privacy, federated learning is proposed
in machine learning as a framework for training deep learning models on the
network edge without sharing data between participating nodes. In this work, we
present an interior point method (IPM) to solve a general ERM problem under the
federated learning setting. We show that the communication complexity of each
iteration of our IPM is , where is the dimension (i.e.,
number of features) of the dataset
Sparse Nonnegative Convolution Is Equivalent to Dense Nonnegative Convolution
Computing the convolution of two length- vectors is an ubiquitous computational primitive. Applications range from string problems to Knapsack-type problems, and from 3SUM to All-Pairs Shortest Paths. These applications often come in the form of nonnegative convolution, where the entries of are nonnegative integers. The classical algorithm to compute uses the Fast Fourier Transform and runs in time . However, often and satisfy sparsity conditions, and hence one could hope for significant improvements. The ideal goal is an -time algorithm, where is the number of non-zero elements in the output, i.e., the size of the support of . This problem is referred to as sparse nonnegative convolution, and has received considerable attention in the literature; the fastest algorithms to date run in time . The main result of this paper is the first -time algorithm for sparse nonnegative convolution. Our algorithm is randomized and assumes that the length and the largest entry of and are subexponential in . Surprisingly, we can phrase our algorithm as a reduction from the sparse case to the dense case of nonnegative convolution, showing that, under some mild assumptions, sparse nonnegative convolution is equivalent to dense nonnegative convolution for constant-error randomized algorithms. Specifically, if is the time to convolve two nonnegative length- vectors with success probability , and is the time to convolve two nonnegative vectors with output size with success probability , then . Our approach uses a variety of new techniques in combination with some old machinery from linear sketching and structured linear algebra, as well as new insights on linear hashing, the most classical hash function
Traversing the FFT Computation Tree for Dimension-Independent Sparse Fourier Transforms
We consider the well-studied Sparse Fourier transform problem, where one aims
to quickly recover an approximately Fourier -sparse vector from observing its time domain representation . In the
exact -sparse case the best known dimension-independent algorithm runs in
near cubic time in and it is unclear whether a faster algorithm like in low
dimensions is possible. Beyond that, all known approaches either suffer from an
exponential dependence on the dimension or can only tolerate a trivial
amount of noise. This is in sharp contrast with the classical FFT of Cooley and
Tukey, which is stable and completely insensitive to the dimension of the input
vector: its runtime is in any dimension for . Our work
aims to address the above issues.
First, we provide a translation/reduction of the exactly -sparse FT
problem to a concrete tree exploration task which asks to recover leaves in
a full binary tree under certain exploration rules. Subsequently, we provide
(a) an almost quadratic in time algorithm for this task, and (b) evidence
that a strongly subquadratic time for Sparse FT via this approach is likely
impossible. We achieve the latter by proving a conditional quadratic time lower
bound on sparse polynomial multipoint evaluation (the classical non-equispaced
sparse FT) which is a core routine in the aforementioned translation. Thus, our
results combined can be viewed as an almost complete understanding of this
approach, which is the only known approach that yields sublinear time
dimension-independent Sparse FT algorithms.
Subsequently, we provide a robustification of our algorithm, yielding a
robust cubic time algorithm under bounded noise. This requires proving
new structural properties of the recently introduced adaptive aliasing filters
combined with a variety of new techniques and ideas
Sparse {Fourier Transform} by Traversing {Cooley-Tukey FFT} Computation Graphs
Computing the dominant Fourier coefficients of a vector is a common task in many fields, such as signal processing, learning theory, and computational complexity. In the Sparse Fast Fourier Transform (Sparse FFT) problem, one is given oracle access to a -dimensional vector of size , and is asked to compute the best -term approximation of its Discrete Fourier Transform, quickly and using few samples of the input vector . While the sample complexity of this problem is quite well understood, all previous approaches either suffer from an exponential dependence of runtime on the dimension or can only tolerate a trivial amount of noise. This is in sharp contrast with the classical FFT algorithm of Cooley and Tukey, which is stable and completely insensitive to the dimension of the input vector: its runtime is in any dimension . In this work, we introduce a new high-dimensional Sparse FFT toolkit and use it to obtain new algorithms, both on the exact, as well as in the case of bounded noise. This toolkit includes i) a new strategy for exploring a pruned FFT computation tree that reduces the cost of filtering, ii) new structural properties of adaptive aliasing filters recently introduced by Kapralov, Velingker and Zandieh'SODA'19, and iii) a novel lazy estimation argument, suited to reducing the cost of estimation in FFT tree-traversal approaches. Our robust algorithm can be viewed as a highly optimized sparse, stable extension of the Cooley-Tukey FFT algorithm. Finally, we explain the barriers we have faced by proving a conditional quadratic lower bound on the running time of the well-studied non-equispaced Fourier transform problem. This resolves a natural and frequently asked question in computational Fourier transforms. Lastly, we provide a preliminary experimental evaluation comparing the runtime of our algorithm to FFTW and SFFT 2.0