58 research outputs found
Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization
We consider the problem of sparse coding, where each sample consists of a
sparse linear combination of a set of dictionary atoms, and the task is to
learn both the dictionary elements and the mixing coefficients. Alternating
minimization is a popular heuristic for sparse coding, where the dictionary and
the coefficients are estimated in alternate steps, keeping the other fixed.
Typically, the coefficients are estimated via minimization, keeping
the dictionary fixed, and the dictionary is estimated through least squares,
keeping the coefficients fixed. In this paper, we establish local linear
convergence for this variant of alternating minimization and establish that the
basin of attraction for the global optimum (corresponding to the true
dictionary and the coefficients) is \order{1/s^2}, where is the sparsity
level in each sample and the dictionary satisfies RIP. Combined with the recent
results of approximate dictionary estimation, this yields provable guarantees
for exact recovery of both the dictionary elements and the coefficients, when
the dictionary elements are incoherent.Comment: Local linear convergence now holds under RIP and also more general
restricted eigenvalue condition
Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization
We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficients are estimated in alternate steps, keeping the other fixed. Typically, the coefficients are estimated via â„“_1 minimization, keeping the dictionary fixed, and the dictionary is estimated through least squares, keeping the coefficients fixed. In this paper, we establish local linear convergence for this variant of alternating minimization and establish that the basin of attraction for the global optimum (corresponding to the true dictionary and the coefficients) is O(1/s^2), where s is the sparsity level in each sample and the dictionary satisfies restricted isometry property. Combined with the recent results of approximate dictionary estimation, this yields provable guarantees for exact recovery of both the dictionary elements and the coefficients, when the dictionary elements are incoherent
Chiron: A Robust Recommendation System with Graph Regularizer
Recommendation systems have been widely used by commercial service providers
for giving suggestions to users. Collaborative filtering (CF) systems, one of
the most popular recommendation systems, utilize the history of behaviors of
the aggregate user-base to provide individual recommendations and are effective
when almost all users faithfully express their opinions. However, they are
vulnerable to malicious users biasing their inputs in order to change the
overall ratings of a specific group of items. CF systems largely fall into two
categories - neighborhood-based and (matrix) factorization-based - and the
presence of adversarial input can influence recommendations in both categories,
leading to instabilities in estimation and prediction. Although the robustness
of different collaborative filtering algorithms has been extensively studied,
designing an efficient system that is immune to manipulation remains a
significant challenge. In this work we propose a novel "hybrid" recommendation
system with an adaptive graph-based user/item similarity-regularization -
"Chiron". Chiron ties the performance benefits of dimensionality reduction
(through factorization) with the advantage of neighborhood clustering (through
regularization). We demonstrate, using extensive comparative experiments, that
Chiron is resistant to manipulation by large and lethal attacks
New Guarantees for Blind Compressed Sensing
Blind Compressed Sensing (BCS) is an extension of Compressed Sensing (CS)
where the optimal sparsifying dictionary is assumed to be unknown and subject
to estimation (in addition to the CS sparse coefficients). Since the emergence
of BCS, dictionary learning, a.k.a. sparse coding, has been studied as a matrix
factorization problem where its sample complexity, uniqueness and
identifiability have been addressed thoroughly. However, in spite of the strong
connections between BCS and sparse coding, recent results from the sparse
coding problem area have not been exploited within the context of BCS. In
particular, prior BCS efforts have focused on learning constrained and complete
dictionaries that limit the scope and utility of these efforts. In this paper,
we develop new theoretical bounds for perfect recovery for the general
unconstrained BCS problem. These unconstrained BCS bounds cover the case of
overcomplete dictionaries, and hence, they go well beyond the existing BCS
theory. Our perfect recovery results integrate the combinatorial theories of
sparse coding with some of the recent results from low-rank matrix recovery. In
particular, we propose an efficient CS measurement scheme that results in
practical recovery bounds for BCS. Moreover, we discuss the performance of BCS
under polynomial-time sparse coding algorithms.Comment: To appear in the 53rd Annual Allerton Conference on Communication,
Control and Computing, University of Illinois at Urbana-Champaign, IL, USA,
201
- …