32 research outputs found

    Optimal Query Complexities for Dynamic Trace Estimation

    Full text link
    We consider the problem of minimizing the number of matrix-vector queries needed for accurate trace estimation in the dynamic setting where our underlying matrix is changing slowly, such as during an optimization process. Specifically, for any mm matrices A1,...,AmA_1,...,A_m with consecutive differences bounded in Schatten-11 norm by α\alpha, we provide a novel binary tree summation procedure that simultaneously estimates all mm traces up to ϵ\epsilon error with δ\delta failure probability with an optimal query complexity of O~(mαlog(1/δ)/ϵ+mlog(1/δ))\widetilde{O}\left(m \alpha\sqrt{\log(1/\delta)}/\epsilon + m\log(1/\delta)\right), improving the dependence on both α\alpha and δ\delta from Dharangutte and Musco (NeurIPS, 2021). Our procedure works without additional norm bounds on AiA_i and can be generalized to a bound for the pp-th Schatten norm for p[1,2]p \in [1,2], giving a complexity of O~(mα(log(1/δ)/ϵ)p+mlog(1/δ))\widetilde{O}\left(m \alpha\left(\sqrt{\log(1/\delta)}/\epsilon\right)^p +m \log(1/\delta)\right). By using novel reductions to communication complexity and information-theoretic analyses of Gaussian matrices, we provide matching lower bounds for static and dynamic trace estimation in all relevant parameters, including the failure probability. Our lower bounds (1) give the first tight bounds for Hutchinson's estimator in the matrix-vector product model with Frobenius norm error even in the static setting, and (2) are the first unconditional lower bounds for dynamic trace estimation, resolving open questions of prior work.Comment: 30 page

    Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems

    Get PDF
    We consider the general problem of learning about a matrix through vector-matrix-vector queries. These queries provide the value of u^{T}Mv over a fixed field ? for a specified pair of vectors u,v ? ??. To motivate these queries, we observe that they generalize many previously studied models, such as independent set queries, cut queries, and standard graph queries. They also specialize the recently studied matrix-vector query model. Our work is exploratory and broad, and we provide new upper and lower bounds for a wide variety of problems, spanning linear algebra, statistics, and graphs. Many of our results are nearly tight, and we use diverse techniques from linear algebra, randomized algorithms, and communication complexity

    From Oja's Algorithm to the Multiplicative Weights Update Method with Applications

    Full text link
    Oja's algorithm is a well known online algorithm studied mainly in the context of stochastic principal component analysis. We make a simple observation, yet to the best of our knowledge a novel one, that when applied to a any (not necessarily stochastic) sequence of symmetric matrices which share common eigenvectors, the regret of Oja's algorithm could be directly bounded in terms of the regret of the well known multiplicative weights update method for the problem of prediction with expert advice. Several applications to optimization with quadratic forms over the unit sphere in Rn\reals^n are discussed

    Testing Positive Semidefiniteness Using Linear Measurements

    Full text link
    We study the problem of testing whether a symmetric d×dd \times d input matrix AA is symmetric positive semidefinite (PSD), or is ϵ\epsilon-far from the PSD cone, meaning that λmin(A)ϵAp\lambda_{\min}(A) \leq - \epsilon \|A\|_p, where Ap\|A\|_p is the Schatten-pp norm of AA. In applications one often needs to quickly tell if an input matrix is PSD, and a small distance from the PSD cone may be tolerable. We consider two well-studied query models for measuring efficiency, namely, the matrix-vector and vector-matrix-vector query models. We first consider one-sided testers, which are testers that correctly classify any PSD input, but may fail on a non-PSD input with a tiny failure probability. Up to logarithmic factors, in the matrix-vector query model we show a tight Θ~(1/ϵp/(2p+1))\widetilde{\Theta}(1/\epsilon^{p/(2p+1)}) bound, while in the vector-matrix-vector query model we show a tight Θ~(d11/p/ϵ)\widetilde{\Theta}(d^{1-1/p}/\epsilon) bound, for every p1p \geq 1. We also show a strong separation between one-sided and two-sided testers in the vector-matrix-vector model, where a two-sided tester can fail on both PSD and non-PSD inputs with a tiny failure probability. In particular, for the important case of the Frobenius norm, we show that any one-sided tester requires Ω~(d/ϵ)\widetilde{\Omega}(\sqrt{d}/\epsilon) queries. However we introduce a bilinear sketch for two-sided testing from which we construct a Frobenius norm tester achieving the optimal O~(1/ϵ2)\widetilde{O}(1/\epsilon^2) queries. We also give a number of additional separations between adaptive and non-adaptive testers. Our techniques have implications beyond testing, providing new methods to approximate the spectrum of a matrix with Frobenius norm error using dimensionality reduction in a way that preserves the signs of eigenvalues

    Query lower bounds for log-concave sampling

    Full text link
    Log-concave sampling has witnessed remarkable algorithmic advances in recent years, but the corresponding problem of proving lower bounds for this task has remained elusive, with lower bounds previously known only in dimension one. In this work, we establish the following query lower bounds: (1) sampling from strongly log-concave and log-smooth distributions in dimension d2d\ge 2 requires Ω(logκ)\Omega(\log \kappa) queries, which is sharp in any constant dimension, and (2) sampling from Gaussians in dimension dd (hence also from general log-concave and log-smooth distributions in dimension dd) requires Ω~(min(κlogd,d))\widetilde \Omega(\min(\sqrt\kappa \log d, d)) queries, which is nearly sharp for the class of Gaussians. Here κ\kappa denotes the condition number of the target distribution. Our proofs rely upon (1) a multiscale construction inspired by work on the Kakeya conjecture in harmonic analysis, and (2) a novel reduction that demonstrates that block Krylov algorithms are optimal for this problem, as well as connections to lower bound techniques based on Wishart matrices developed in the matrix-vector query literature.Comment: 46 pages, 2 figure
    corecore