We consider the problem of minimizing the number of matrix-vector queries
needed for accurate trace estimation in the dynamic setting where our
underlying matrix is changing slowly, such as during an optimization process.
Specifically, for any m matrices A1,...,Am with consecutive differences
bounded in Schatten-1 norm by α, we provide a novel binary tree
summation procedure that simultaneously estimates all m traces up to
ϵ error with δ failure probability with an optimal query
complexity of O(mαlog(1/δ)/ϵ+mlog(1/δ)), improving the dependence on both α and δ
from Dharangutte and Musco (NeurIPS, 2021). Our procedure works without
additional norm bounds on Ai and can be generalized to a bound for the
p-th Schatten norm for p∈[1,2], giving a complexity of
O(mα(log(1/δ)/ϵ)p+mlog(1/δ)).
By using novel reductions to communication complexity and
information-theoretic analyses of Gaussian matrices, we provide matching lower
bounds for static and dynamic trace estimation in all relevant parameters,
including the failure probability. Our lower bounds (1) give the first tight
bounds for Hutchinson's estimator in the matrix-vector product model with
Frobenius norm error even in the static setting, and (2) are the first
unconditional lower bounds for dynamic trace estimation, resolving open
questions of prior work.Comment: 30 page