24 research outputs found
(Nearly) Sample-Optimal Sparse Fourier Transform in Any Dimension; RIPless and Filterless
In this paper, we consider the extensively studied problem of computing a
-sparse approximation to the -dimensional Fourier transform of a length
signal. Our algorithm uses samples, is dimension-free,
operates for any universe size, and achieves the strongest
guarantee, while running in a time comparable to the Fast Fourier Transform. In
contrast to previous algorithms which proceed either via the Restricted
Isometry Property or via filter functions, our approach offers a fresh
perspective to the sparse Fourier Transform problem
A hybrid Fourier-Prony method
The FFT algorithm that implements the discrete Fourier transform is
considered one of the top ten algorithms of the th century. Its main
strengths are the low computational cost of ) and its
stability. It is one of the most commonly used algorithms to analyze signals
with a dense frequency representation. In recent years there has been an
increasing interest in sparse signal representations and a need for algorithms
that exploit such structure. We propose a new technique that combines the
properties of the discrete Fourier transform with the sparsity of the signal.
This is achieved by integrating ideas of Prony's method into Fourier's method.
The resulting technique has the same frequency resolution as the original FFT
algorithm but uses fewer samples and can achieve a lower computational cost.
Moreover, the proposed algorithm is well suited for a parallel implementation
Thwarting Adversarial Examples: An -RobustSparse Fourier Transform
We give a new algorithm for approximating the Discrete Fourier transform of
an approximately sparse signal that has been corrupted by worst-case
noise, namely a bounded number of coordinates of the signal have been corrupted
arbitrarily. Our techniques generalize to a wide range of linear
transformations that are used in data analysis such as the Discrete Cosine and
Sine transforms, the Hadamard transform, and their high-dimensional analogs. We
use our algorithm to successfully defend against well known adversaries
in the setting of image classification. We give experimental results on the
Jacobian-based Saliency Map Attack (JSMA) and the Carlini Wagner (CW)
attack on the MNIST and Fashion-MNIST datasets as well as the Adversarial Patch
on the ImageNet dataset.Comment: Accepted at 32nd Conference on Neural Information Processing Systems
(NeurIPS 2018), Montr\'eal, Canad
Solving Empirical Risk Minimization in the Current Matrix Multiplication Time
Many convex problems in machine learning and computer science share the same
form: \begin{align*} \min_{x} \sum_{i} f_i( A_i x + b_i), \end{align*} where
are convex functions on with constant , , and .
This problem generalizes linear programming and includes many problems in
empirical risk minimization. In this paper, we give an algorithm that runs in
time \begin{align*} O^* ( ( n^{\omega} + n^{2.5 - \alpha/2} + n^{2+ 1/6} ) \log
(n / \delta) ) \end{align*} where is the exponent of matrix
multiplication, is the dual exponent of matrix multiplication, and
is the relative accuracy. Note that the runtime has only a log
dependence on the condition numbers or other data dependent parameters and
these are captured in . For the current bound
[Vassilevska Williams'12, Le Gall'14] and [Le Gall,
Urrutia'18], our runtime matches the
current best for solving a dense least squares regression problem, a special
case of the problem we consider. Very recently, [Alman'18] proved that all the
current known techniques can not give a better below which is
larger than our . Our result generalizes the very recent result of
solving linear programs in the current matrix multiplication time [Cohen, Lee,
Song'19] to a more broad class of problems. Our algorithm proposes two concepts
which are different from [Cohen, Lee, Song'19] :
We give a robust deterministic central path method, whereas the
previous one is a stochastic central path which updates weights by a random
sparse vector.
We propose an efficient data-structure to maintain the central path
of interior point methods even when the weights update vector is dense
Tighter Fourier Transform Complexity Tradeoffs
The Fourier Transform is one of the most important linear transformations
used in science and engineering. Cooley and Tukey's Fast Fourier Transform
(FFT) from 1964 is a method for computing this transformation in time . Achieving a matching lower bound in a reasonable computational model is
one of the most important open problems in theoretical computer science.
In 2014, improving on his previous work, Ailon showed that if an algorithm
speeds up the FFT by a factor of , then it must rely on
computing, as an intermediate "bottleneck" step, a linear mapping of the input
with condition number . Our main result shows that a factor
speedup implies existence of not just one but -ill conditioned
bottlenecks occurring at different steps, each causing information
from independent (orthogonal) components of the input to either overflow or
underflow. This provides further evidence that beating FFT is hard. Our result
also gives the first quantitative tradeoff between computation speed and
information loss in Fourier computation on fixed word size architectures. The
main technical result is an entropy analysis of the Fourier transform under
transformations of low trace, which is interesting in its own right
Learning Long Term Dependencies via Fourier Recurrent Units
It is a known fact that training recurrent neural networks for tasks that
have long term dependencies is challenging. One of the main reasons is the
vanishing or exploding gradient problem, which prevents gradient information
from propagating to early layers. In this paper we propose a simple recurrent
architecture, the Fourier Recurrent Unit (FRU), that stabilizes the gradients
that arise in its training while giving us stronger expressive power.
Specifically, FRU summarizes the hidden states along the temporal
dimension with Fourier basis functions. This allows gradients to easily reach
any layer due to FRU's residual learning structure and the global support of
trigonometric functions. We show that FRU has gradient lower and upper bounds
independent of temporal dimension. We also show the strong expressivity of
sparse Fourier basis, from which FRU obtains its strong expressive power. Our
experimental study also demonstrates that with fewer parameters the proposed
architecture outperforms other recurrent architectures on many tasks
Fast and Efficient Sparse 2D Discrete Fourier Transform using Sparse-Graph Codes
We present a novel algorithm, named the 2D-FFAST, to compute a sparse
2D-Discrete Fourier Transform (2D-DFT) featuring both low sample complexity and
low computational complexity. The proposed algorithm is based on mixed concepts
from signal processing (sub-sampling and aliasing), coding theory (sparse-graph
codes) and number theory (Chinese-remainder-theorem) and generalizes the
1D-FFAST 2 algorithm recently proposed by Pawar and Ramchandran [1] to the 2D
setting. Concretely, our proposed 2D-FFAST algorithm computes a k-sparse
2D-DFT, with a uniformly random support, of size N = Nx x Ny using O(k)
noiseless spatial-domain measurements in O(k log k) computational time. Our
results are attractive when the sparsity is sub-linear with respect to the
signal dimension, that is, when k -> infinity and k/N -> 0. For the case when
the spatial-domain measurements are corrupted by additive noise, our 2D-FFAST
framework extends to a noise-robust version in sub-linear time of O(k log4 N )
using O(k log3 N ) measurements. Simulation results, on synthetic images as
well as real-world magnetic resonance images, are provided in Section VII and
demonstrate the empirical performance of the proposed 2D-FFAST algorithm
A sparse multidimensional FFT for real positive vectors
We present a sparse multidimensional FFT (sMFFT) randomized algorithm for
real positive vectors. The algorithm works in any fixed dimension, requires
(O(R log(R) log(N)) ) samples and runs in O( R log^2(R) log(N)) complexity
(where N is the total size of the vector in d dimensions and R is the number of
nonzeros). It is stable to low-level noise and exhibits an exponentially small
probability of failure.Comment: Fixed minor typos. Corrected use of Q^{-1} in Algorithm 3 and theore
Deterministic Sparse Fourier Transform with an ell_infty Guarantee
In this paper we revisit the deterministic version of the Sparse Fourier
Transform problem, which asks to read only a few entries of and design a recovery algorithm such that the output of the
algorithm approximates , the Discrete Fourier Transform (DFT) of .
The randomized case has been well-understood, while the main work in the
deterministic case is that of Merhi et al.\@ (J Fourier Anal Appl 2018), which
obtains samples and a similar runtime
with the guarantee. We focus on the stronger
guarantee and the closely related problem of incoherent
matrices. We list our contributions as follows.
1. We find a deterministic collection of samples for the
recovery in time , and a deterministic
collection of samples for the sparse
recovery in time .
2. We give new deterministic constructions of incoherent matrices that are
row-sampled submatrices of the DFT matrix, via a derandomization of Bernstein's
inequality and bounds on exponential sums considered in analytic number theory.
Our first construction matches a previous randomized construction of Nelson,
Nguyen and Woodruff (RANDOM'12), where there was no constraint on the form of
the incoherent matrix.
Our algorithms are nearly sample-optimal, since a lower bound of is known, even for the case where the sensing matrix can be
arbitrarily designed. A similar lower bound of is
known for incoherent matrices.Comment: ICALP 2020--presentation improved according to reviewers' comment
Stronger L2/L2 Compressed Sensing; Without Iterating
We consider the extensively studied problem of compressed
sensing. The main contribution of our work is an improvement over [Gilbert, Li,
Porat and Strauss, STOC 2010] with faster decoding time and significantly
smaller column sparsity, answering two open questions of the aforementioned
work.
Previous work on sublinear-time compressed sensing employed an iterative
procedure, recovering the heavy coordinates in phases. We completely depart
from that framework, and give the first sublinear-time scheme
which achieves the optimal number of measurements without iterating; this new
approach is the key step to our progress. Towards that, we satisfy the
guarantee by exploiting the heaviness of coordinates in a way
that was not exploited in previous work. Via our techniques we obtain improved
results for various sparse recovery tasks, and indicate possible further
applications to problems in the field, to which the aforementioned iterative
procedure creates significant obstructions