255,078 research outputs found
Unified View of Matrix Completion under General Structural Constraints
In this paper, we present a unified analysis of matrix completion under
general low-dimensional structural constraints induced by {\em any} norm
regularization. We consider two estimators for the general problem of
structured matrix completion, and provide unified upper bounds on the sample
complexity and the estimation error. Our analysis relies on results from
generic chaining, and we establish two intermediate results of independent
interest: (a) in characterizing the size or complexity of low dimensional
subsets in high dimensional ambient space, a certain partial complexity measure
encountered in the analysis of matrix completion problems is characterized in
terms of a well understood complexity measure of Gaussian widths, and (b) it is
shown that a form of restricted strong convexity holds for matrix completion
problems under general norm regularization. Further, we provide several
non-trivial examples of structures included in our framework, notably the
recently proposed spectral -support norm.Comment: published in NIPS 2015. Advances in Neural Information Processing
Systems 28, 201
Sparse Group Inductive Matrix Completion
We consider the problem of matrix completion with side information
(\textit{inductive matrix completion}). In real-world applications many
side-channel features are typically non-informative making feature selection an
important part of the problem. We incorporate feature selection into inductive
matrix completion by proposing a matrix factorization framework with
group-lasso regularization on side feature parameter matrices. We demonstrate,
that the theoretical sample complexity for the proposed method is much lower
compared to its competitors in sparse problems, and propose an efficient
optimization algorithm for the resulting low-rank matrix completion problem
with sparsifying regularizers. Experiments on synthetic and real-world datasets
show that the proposed approach outperforms other methods
Accelerated and Inexact Soft-Impute for Large-Scale Matrix and Tensor Completion
Matrix and tensor completion aim to recover a low-rank matrix / tensor from
limited observations and have been commonly used in applications such as
recommender systems and multi-relational data mining. A state-of-the-art matrix
completion algorithm is Soft-Impute, which exploits the special "sparse plus
low-rank" structure of the matrix iterates to allow efficient SVD in each
iteration. Though Soft-Impute is a proximal algorithm, it is generally believed
that acceleration destroys the special structure and is thus not useful. In
this paper, we show that Soft-Impute can indeed be accelerated without
comprising this structure. To further reduce the iteration time complexity, we
propose an approximate singular value thresholding scheme based on the power
method. Theoretical analysis shows that the proposed algorithm still enjoys the
fast convergence rate of accelerated proximal algorithms. We further
extend the proposed algorithm to tensor completion with the scaled latent
nuclear norm regularizer. We show that a similar "sparse plus low-rank"
structure also exists, leading to low iteration complexity and fast
convergence rate. Extensive experiments demonstrate that the proposed algorithm
is much faster than Soft-Impute and other state-of-the-art matrix and tensor
completion algorithms.Comment: Journal version of previous conference paper 'Accelerated inexact
soft-impute for fast large-scale matrix completion' appeared at IJCAI 201
Recommendation via matrix completion using Kolmogorov complexity
A usual way to model a recommendation system is as a matrix completion
problem. There are several matrix completion methods, typically using
optimization approaches or collaborative filtering. Most approaches assume that
the matrix is either low rank, or that there are a small number of latent
variables that encode the full problem. Here, we propose a novel matrix
completion algorithm for recommendation systems, without any assumptions on the
rank and that is model free, i.e., the entries are not assumed to be a function
of some latent variables. Instead, we use a technique akin to information
theory. Our method performs hybrid neighborhood-based collaborative filtering
using Kolmogorov complexity. It decouples the matrix completion into a vector
completion problem for each user. The recommendation for one user is thus
independent of the recommendation for other users. This makes the algorithm
scalable because the computations are highly parallelizable. Our results are
competitive with state-of-the-art approaches on both synthetic and real-world
dataset benchmarks.Comment: 9 pages, 1 figure, 3 table
A Geometric Approach to Low-Rank Matrix Completion
The low-rank matrix completion problem can be succinctly stated as follows:
given a subset of the entries of a matrix, find a low-rank matrix consistent
with the observations. While several low-complexity algorithms for matrix
completion have been proposed so far, it remains an open problem to devise
search procedures with provable performance guarantees for a broad class of
matrix models. The standard approach to the problem, which involves the
minimization of an objective function defined using the Frobenius metric, has
inherent difficulties: the objective function is not continuous and the
solution set is not closed. To address this problem, we consider an
optimization procedure that searches for a column (or row) space that is
geometrically consistent with the partial observations. The geometric objective
function is continuous everywhere and the solution set is the closure of the
solution set of the Frobenius metric. We also preclude the existence of local
minimizers, and hence establish strong performance guarantees, for special
completion scenarios, which do not require matrix incoherence or large matrix
size.Comment: 10 pages, 2 figure
Computational Limits for Matrix Completion
Matrix Completion is the problem of recovering an unknown real-valued
low-rank matrix from a subsample of its entries. Important recent results show
that the problem can be solved efficiently under the assumption that the
unknown matrix is incoherent and the subsample is drawn uniformly at random.
Are these assumptions necessary?
It is well known that Matrix Completion in its full generality is NP-hard.
However, little is known if make additional assumptions such as incoherence and
permit the algorithm to output a matrix of slightly higher rank. In this paper
we prove that Matrix Completion remains computationally intractable even if the
unknown matrix has rank but we are allowed to output any constant rank
matrix, and even if additionally we assume that the unknown matrix is
incoherent and are shown of the entries. This result relies on the
conjectured hardness of the -Coloring problem. We also consider the positive
semidefinite Matrix Completion problem. Here we show a similar hardness result
under the standard assumption that
Our results greatly narrow the gap between existing feasibility results and
computational lower bounds. In particular, we believe that our results give the
first complexity-theoretic justification for why distributional assumptions are
needed beyond the incoherence assumption in order to obtain positive results.
On the technical side, we contribute several new ideas on how to encode hard
combinatorial problems in low-rank optimization problems. We hope that these
techniques will be helpful in further understanding the computational limits of
Matrix Completion and related problems
Algebraic Variety Models for High-Rank Matrix Completion
We consider a generalization of low-rank matrix completion to the case where
the data belongs to an algebraic variety, i.e. each data point is a solution to
a system of polynomial equations. In this case the original matrix is possibly
high-rank, but it becomes low-rank after mapping each column to a higher
dimensional space of monomial features. Many well-studied extensions of linear
models, including affine subspaces and their union, can be described by a
variety model. In addition, varieties can be used to model a richer class of
nonlinear quadratic and higher degree curves and surfaces. We study the
sampling requirements for matrix completion under a variety model with a focus
on a union of affine subspaces. We also propose an efficient matrix completion
algorithm that minimizes a convex or non-convex surrogate of the rank of the
matrix of monomial features. Our algorithm uses the well-known "kernel trick"
to avoid working directly with the high-dimensional monomial matrix. We show
the proposed algorithm is able to recover synthetically generated data up to
the predicted sampling complexity bounds. The proposed algorithm also
outperforms standard low rank matrix completion and subspace clustering
techniques in experiments with real data
Symmetric Tensor Completion from Multilinear Entries and Learning Product Mixtures over the Hypercube
We give an algorithm for completing an order- symmetric low-rank tensor
from its multilinear entries in time roughly proportional to the number of
tensor entries. We apply our tensor completion algorithm to the problem of
learning mixtures of product distributions over the hypercube, obtaining new
algorithmic results. If the centers of the product distribution are linearly
independent, then we recover distributions with as many as centers
in polynomial time and sample complexity. In the general case, we recover
distributions with as many as centers in quasi-polynomial
time, answering an open problem of Feldman et al. (SIAM J. Comp.) for the
special case of distributions with incoherent bias vectors.
Our main algorithmic tool is the iterated application of a low-rank matrix
completion algorithm for matrices with adversarially missing entries.Comment: Removed adversarial matrix completion algorithm after discovering
that our matrix completion results can be derived from prior wor
Matrix Completion Methods for Causal Panel Data Models
In this paper we study methods for estimating causal effects in settings with
panel data, where some units are exposed to a treatment during some periods and
the goal is estimating counterfactual (untreated) outcomes for the treated
unit/period combinations. We develop a class of matrix completion estimators
that uses the observed elements of the matrix of control outcomes corresponding
to untreated unit/periods to impute the "missing" elements of the control
outcome matrix, corresponding to treated units/periods. The approach estimates
a matrix that well-approximates the original (incomplete) matrix, but has lower
complexity according to the nuclear norm for matrices. We generalize results
from the matrix completion literature by allowing the patterns of missing data
to have a time series dependency structure. We present novel insights
concerning the connections between the matrix completion literature, the
literature on interactive fixed effects models and the literatures on program
evaluation under unconfoundedness and synthetic control methods. We show that
all these estimators can be viewed as focusing on the same objective function.
They differ in the way they deal with lack of identification, in some cases
solely through regularization (our proposed nuclear norm matrix completion
estimator) and in other cases primarily through imposing hard restrictions (the
unconfoundedness and synthetic control approaches). proposed method outperforms
unconfoundedness-based or synthetic control estimators
Sample Complexity of Power System State Estimation using Matrix Completion
In this paper, we propose an analytical framework to quantify the amount of
data samples needed to obtain accurate state estimation in a power system - a
problem known as sample complexity analysis in computer science. Motivated by
the increasing adoption of distributed energy resources into the
distribution-level grids, it becomes imperative to estimate the state of
distribution grids in order to ensure stable operation. Traditional power
system state estimation techniques mainly focus on the transmission network
which involve solving an overdetermined system and eliminating bad data.
However, distribution networks are typically underdetermined due to the large
number of connection points and high cost of pervasive installation of
measurement devices. In this paper, we consider the recently proposed
state-estimation method for underdetermined systems that is based on matrix
completion. In particular, a constrained matrix completion algorithm was
proposed, wherein the standard matrix completion problem is augmented with
additional equality constraints representing the physics (namely power-flow
constraints). We analyze the sample complexity of this general method by
proving an upper bound on the sample complexity that depends directly on the
properties of these constraints that can lower number of needed samples as
compared to the unconstrained problem. To demonstrate the improvement that the
constraints add to distribution state estimation, we test the method on a
141-bus distribution network case study and compare it to the traditional least
squares minimization state estimation method
- …