2 research outputs found
Near-Optimal Estimation of Simultaneously Sparse and Low-Rank Matrices from Nested Linear Measurements
In this paper we consider the problem of estimating simultaneously low-rank
and row-wise sparse matrices from nested linear measurements where the linear
operator consists of the product of a linear operator and a
matrix . Leveraging the nested structure of the measurement
operator, we propose a computationally efficient two-stage algorithm for
estimating the simultaneously structured target matrix. Assuming that
is a restricted isometry for low-rank matrices and
is a restricted isometry for row-wise sparse matrices, we
establish an accuracy guarantee that holds uniformly for all sufficiently
low-rank and row-wise sparse matrices with high probability. Furthermore, using
standard tools from information theory, we establish a minimax lower bound for
estimation of simultaneously low-rank and row-wise sparse matrices from linear
measurements that need not be nested. The accuracy bounds established for the
algorithm, that also serve as a minimax upper bound, differ from the derived
minimax lower bound merely by a polylogarithmic factor of the dimensions.
Therefore, the proposed algorithm is nearly minimax optimal. We also discuss
some applications of the proposed observation model and evaluate our algorithm
through numerical simulation
Active Sampling Count Sketch (ASCS) for Online Sparse Estimation of a Trillion Scale Covariance Matrix
Estimating and storing the covariance (or correlation) matrix of
high-dimensional data is computationally challenging because both memory and
computational requirements scale quadratically with the dimension. Fortunately,
high-dimensional covariance matrices as observed in text, click-through,
meta-genomics datasets, etc are often sparse. In this paper, we consider the
problem of efficient sparse estimation of covariance matrices with possibly
trillions of entries. The size of the datasets we target requires the algorithm
to be online, as more than one pass over the data is prohibitive. In this
paper, we propose Active Sampling Count Sketch (ASCS), an online and one-pass
sketching algorithm, that recovers the large entries of the covariance matrix
accurately. Count Sketch (CS), and other sub-linear compressed sensing
algorithms, offer a natural solution to the problem in theory. However, vanilla
CS does not work well in practice due to a low signal-to-noise ratio (SNR). At
the heart of our approach is a novel active sampling strategy that increases
the SNR of classical CS. We demonstrate the practicality of our algorithm with
synthetic data and real-world high dimensional datasets. ASCS significantly
improves over vanilla CS, demonstrating the merit of our active sampling
strategy.Comment: 13 page