32 research outputs found
Optimal Query Complexities for Dynamic Trace Estimation
We consider the problem of minimizing the number of matrix-vector queries
needed for accurate trace estimation in the dynamic setting where our
underlying matrix is changing slowly, such as during an optimization process.
Specifically, for any matrices with consecutive differences
bounded in Schatten- norm by , we provide a novel binary tree
summation procedure that simultaneously estimates all traces up to
error with failure probability with an optimal query
complexity of , improving the dependence on both and
from Dharangutte and Musco (NeurIPS, 2021). Our procedure works without
additional norm bounds on and can be generalized to a bound for the
-th Schatten norm for , giving a complexity of
.
By using novel reductions to communication complexity and
information-theoretic analyses of Gaussian matrices, we provide matching lower
bounds for static and dynamic trace estimation in all relevant parameters,
including the failure probability. Our lower bounds (1) give the first tight
bounds for Hutchinson's estimator in the matrix-vector product model with
Frobenius norm error even in the static setting, and (2) are the first
unconditional lower bounds for dynamic trace estimation, resolving open
questions of prior work.Comment: 30 page
Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems
We consider the general problem of learning about a matrix through vector-matrix-vector queries. These queries provide the value of u^{T}Mv over a fixed field ? for a specified pair of vectors u,v ? ??. To motivate these queries, we observe that they generalize many previously studied models, such as independent set queries, cut queries, and standard graph queries. They also specialize the recently studied matrix-vector query model. Our work is exploratory and broad, and we provide new upper and lower bounds for a wide variety of problems, spanning linear algebra, statistics, and graphs. Many of our results are nearly tight, and we use diverse techniques from linear algebra, randomized algorithms, and communication complexity
From Oja's Algorithm to the Multiplicative Weights Update Method with Applications
Oja's algorithm is a well known online algorithm studied mainly in the
context of stochastic principal component analysis. We make a simple
observation, yet to the best of our knowledge a novel one, that when applied to
a any (not necessarily stochastic) sequence of symmetric matrices which share
common eigenvectors, the regret of Oja's algorithm could be directly bounded in
terms of the regret of the well known multiplicative weights update method for
the problem of prediction with expert advice. Several applications to
optimization with quadratic forms over the unit sphere in are
discussed
Testing Positive Semidefiniteness Using Linear Measurements
We study the problem of testing whether a symmetric input matrix
is symmetric positive semidefinite (PSD), or is -far from the PSD
cone, meaning that , where
is the Schatten- norm of . In applications one often needs to quickly
tell if an input matrix is PSD, and a small distance from the PSD cone may be
tolerable. We consider two well-studied query models for measuring efficiency,
namely, the matrix-vector and vector-matrix-vector query models. We first
consider one-sided testers, which are testers that correctly classify any PSD
input, but may fail on a non-PSD input with a tiny failure probability. Up to
logarithmic factors, in the matrix-vector query model we show a tight
bound, while in the
vector-matrix-vector query model we show a tight
bound, for every . We also
show a strong separation between one-sided and two-sided testers in the
vector-matrix-vector model, where a two-sided tester can fail on both PSD and
non-PSD inputs with a tiny failure probability. In particular, for the
important case of the Frobenius norm, we show that any one-sided tester
requires queries. However we introduce
a bilinear sketch for two-sided testing from which we construct a Frobenius
norm tester achieving the optimal queries. We
also give a number of additional separations between adaptive and non-adaptive
testers. Our techniques have implications beyond testing, providing new methods
to approximate the spectrum of a matrix with Frobenius norm error using
dimensionality reduction in a way that preserves the signs of eigenvalues
Query lower bounds for log-concave sampling
Log-concave sampling has witnessed remarkable algorithmic advances in recent
years, but the corresponding problem of proving lower bounds for this task has
remained elusive, with lower bounds previously known only in dimension one. In
this work, we establish the following query lower bounds: (1) sampling from
strongly log-concave and log-smooth distributions in dimension
requires queries, which is sharp in any constant
dimension, and (2) sampling from Gaussians in dimension (hence also from
general log-concave and log-smooth distributions in dimension ) requires
queries, which is nearly sharp
for the class of Gaussians. Here denotes the condition number of the
target distribution. Our proofs rely upon (1) a multiscale construction
inspired by work on the Kakeya conjecture in harmonic analysis, and (2) a novel
reduction that demonstrates that block Krylov algorithms are optimal for this
problem, as well as connections to lower bound techniques based on Wishart
matrices developed in the matrix-vector query literature.Comment: 46 pages, 2 figure