44,095 research outputs found
High Dimensional Low Rank plus Sparse Matrix Decomposition
This paper is concerned with the problem of low rank plus sparse matrix
decomposition for big data. Conventional algorithms for matrix decomposition
use the entire data to extract the low-rank and sparse components, and are
based on optimization problems with complexity that scales with the dimension
of the data, which limits their scalability. Furthermore, existing randomized
approaches mostly rely on uniform random sampling, which is quite inefficient
for many real world data matrices that exhibit additional structures (e.g.
clustering). In this paper, a scalable subspace-pursuit approach that
transforms the decomposition problem to a subspace learning problem is
proposed. The decomposition is carried out using a small data sketch formed
from sampled columns/rows. Even when the data is sampled uniformly at random,
it is shown that the sufficient number of sampled columns/rows is roughly
O(r\mu), where \mu is the coherency parameter and r the rank of the low rank
component. In addition, adaptive sampling algorithms are proposed to address
the problem of column/row sampling from structured data. We provide an analysis
of the proposed method with adaptive sampling and show that adaptive sampling
makes the required number of sampled columns/rows invariant to the distribution
of the data. The proposed approach is amenable to online implementation and an
online scheme is proposed.Comment: IEEE Transactions on Signal Processin
Random Sampling in Computational Algebra: Helly Numbers and Violator Spaces
This paper transfers a randomized algorithm, originally used in geometric
optimization, to computational problems in commutative algebra. We show that
Clarkson's sampling algorithm can be applied to two problems in computational
algebra: solving large-scale polynomial systems and finding small generating
sets of graded ideals. The cornerstone of our work is showing that the theory
of violator spaces of G\"artner et al.\ applies to polynomial ideal problems.
To show this, one utilizes a Helly-type result for algebraic varieties. The
resulting algorithms have expected runtime linear in the number of input
polynomials, making the ideas interesting for handling systems with very large
numbers of polynomials, but whose rank in the vector space of polynomials is
small (e.g., when the number of variables and degree is constant).Comment: Minor edits, added two references; results unchange
Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method
The block Kaczmarz method is an iterative scheme for solving overdetermined
least-squares problems. At each step, the algorithm projects the current
iterate onto the solution space of a subset of the constraints. This paper
describes a block Kaczmarz algorithm that uses a randomized control scheme to
choose the subset at each step. This algorithm is the first block Kaczmarz
method with an (expected) linear rate of convergence that can be expressed in
terms of the geometric properties of the matrix and its submatrices. The
analysis reveals that the algorithm is most effective when it is given a good
row paving of the matrix, a partition of the rows into well-conditioned blocks.
The operator theory literature provides detailed information about the
existence and construction of good row pavings. Together, these results yield
an efficient block Kaczmarz scheme that applies to many overdetermined
least-squares problem
Online Contention Resolution Schemes
We introduce a new rounding technique designed for online optimization
problems, which is related to contention resolution schemes, a technique
initially introduced in the context of submodular function maximization. Our
rounding technique, which we call online contention resolution schemes (OCRSs),
is applicable to many online selection problems, including Bayesian online
selection, oblivious posted pricing mechanisms, and stochastic probing models.
It allows for handling a wide set of constraints, and shares many strong
properties of offline contention resolution schemes. In particular, OCRSs for
different constraint families can be combined to obtain an OCRS for their
intersection. Moreover, we can approximately maximize submodular functions in
the online settings we consider.
We, thus, get a broadly applicable framework for several online selection
problems, which improves on previous approaches in terms of the types of
constraints that can be handled, the objective functions that can be dealt
with, and the assumptions on the strength of the adversary. Furthermore, we
resolve two open problems from the literature; namely, we present the first
constant-factor constrained oblivious posted price mechanism for matroid
constraints, and the first constant-factor algorithm for weighted stochastic
probing with deadlines.Comment: 33 pages. To appear in SODA 201
- …