11 research outputs found
Lower Complexity Bounds of Finite-Sum Optimization Problems: The Results and Construction
The contribution of this paper includes two aspects. First, we study the
lower bound complexity for the minimax optimization problem whose objective
function is the average of individual smooth component functions. We
consider Proximal Incremental First-order (PIFO) algorithms which have access
to gradient and proximal oracle for each individual component. We develop a
novel approach for constructing adversarial problems, which partitions the
tridiagonal matrix of classical examples into groups. This construction is
friendly to the analysis of incremental gradient and proximal oracle. With this
approach, we demonstrate the lower bounds of first-order algorithms for finding
an -suboptimal point and an -stationary point in
different settings. Second, we also derive the lower bounds of minimization
optimization with PIFO algorithms from our approach, which can cover the
results in \citep{woodworth2016tight} and improve the results in
\citep{zhou2019lower}
Revisiting Co-Occurring Directions: Sharper Analysis and Efficient Algorithm for Sparse Matrices
We study the streaming model for approximate matrix multiplication (AMM). We
are interested in the scenario that the algorithm can only take one pass over
the data with limited memory. The state-of-the-art deterministic sketching
algorithm for streaming AMM is the co-occurring directions (COD), which has
much smaller approximation errors than randomized algorithms and outperforms
other deterministic sketching methods empirically. In this paper, we provide a
tighter error bound for COD whose leading term considers the potential
approximate low-rank structure and the correlation of input matrices. We prove
COD is space optimal with respect to our improved error bound. We also propose
a variant of COD for sparse matrices with theoretical guarantees. The
experiments on real-world sparse datasets show that the proposed algorithm is
more efficient than baseline methods