138,586 research outputs found
1-Bit Matrix Completion
In this paper we develop a theory of matrix completion for the extreme case
of noisy 1-bit observations. Instead of observing a subset of the real-valued
entries of a matrix M, we obtain a small number of binary (1-bit) measurements
generated according to a probability distribution determined by the real-valued
entries of M. The central question we ask is whether or not it is possible to
obtain an accurate estimate of M from this data. In general this would seem
impossible, but we show that the maximum likelihood estimate under a suitable
constraint returns an accurate estimate of M when ||M||_{\infty} <= \alpha, and
rank(M) <= r. If the log-likelihood is a concave function (e.g., the logistic
or probit observation models), then we can obtain this maximum likelihood
estimate by optimizing a convex program. In addition, we also show that if
instead of recovering M we simply wish to obtain an estimate of the
distribution generating the 1-bit measurements, then we can eliminate the
requirement that ||M||_{\infty} <= \alpha. For both cases, we provide lower
bounds showing that these estimates are near-optimal. We conclude with a suite
of experiments that both verify the implications of our theorems as well as
illustrate some of the practical applications of 1-bit matrix completion. In
particular, we compare our program to standard matrix completion methods on
movie rating data in which users submit ratings from 1 to 5. In order to use
our program, we quantize this data to a single bit, but we allow the standard
matrix completion program to have access to the original ratings (from 1 to 5).
Surprisingly, the approach based on binary data performs significantly better
1-Bit Matrix Completion under Exact Low-Rank Constraint
We consider the problem of noisy 1-bit matrix completion under an exact rank
constraint on the true underlying matrix . Instead of observing a subset
of the noisy continuous-valued entries of a matrix , we observe a subset
of noisy 1-bit (or binary) measurements generated according to a probabilistic
model. We consider constrained maximum likelihood estimation of , under a
constraint on the entry-wise infinity-norm of and an exact rank
constraint. This is in contrast to previous work which has used convex
relaxations for the rank. We provide an upper bound on the matrix estimation
error under this model. Compared to the existing results, our bound has faster
convergence rate with matrix dimensions when the fraction of revealed 1-bit
observations is fixed, independent of the matrix dimensions. We also propose an
iterative algorithm for solving our nonconvex optimization with a certificate
of global optimality of the limiting point. This algorithm is based on low rank
factorization of . We validate the method on synthetic and real data with
improved performance over existing methods.Comment: 6 pages, 3 figures, to appear in CISS 201
A Max-Norm Constrained Minimization Approach to 1-Bit Matrix Completion
We consider in this paper the problem of noisy 1-bit matrix completion under
a general non-uniform sampling distribution using the max-norm as a convex
relaxation for the rank. A max-norm constrained maximum likelihood estimate is
introduced and studied. The rate of convergence for the estimate is obtained.
Information-theoretical methods are used to establish a minimax lower bound
under the general sampling model. The minimax upper and lower bounds together
yield the optimal rate of convergence for the Frobenius norm loss.
Computational algorithms and numerical performance are also discussed.Comment: 33 pages, 3 figure
High Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantization
In this paper, we propose a uniformly dithered 1-bit quantization scheme for
high-dimensional statistical estimation. The scheme contains truncation,
dithering, and quantization as typical steps. As canonical examples, the
quantization scheme is applied to the estimation problems of sparse covariance
matrix estimation, sparse linear regression (i.e., compressed sensing), and
matrix completion. We study both sub-Gaussian and heavy-tailed regimes, where
the underlying distribution of heavy-tailed data is assumed to have bounded
moments of some order. We propose new estimators based on 1-bit quantized data.
In sub-Gaussian regime, our estimators achieve near minimax rates, indicating
that our quantization scheme costs very little. In heavy-tailed regime, while
the rates of our estimators become essentially slower, these results are either
the first ones in an 1-bit quantized and heavy-tailed setting, or already
improve on existing comparable results from some respect. Under the
observations in our setting, the rates are almost tight in compressed sensing
and matrix completion. Our 1-bit compressed sensing results feature general
sensing vector that is sub-Gaussian or even heavy-tailed. We also first
investigate a novel setting where both the covariate and response are
quantized. In addition, our approach to 1-bit matrix completion does not rely
on likelihood and represent the first method robust to pre-quantization noise
with unknown distribution. Experimental results on synthetic data are presented
to support our theoretical analysis.Comment: We add lower bounds for 1-bit quantization of heavy-tailed data
(Theorems 11, 14
PU Learning for Matrix Completion
In this paper, we consider the matrix completion problem when the
observations are one-bit measurements of some underlying matrix M, and in
particular the observed samples consist only of ones and no zeros. This problem
is motivated by modern applications such as recommender systems and social
networks where only "likes" or "friendships" are observed. The problem of
learning from only positive and unlabeled examples, called PU
(positive-unlabeled) learning, has been studied in the context of binary
classification. We consider the PU matrix completion problem, where an
underlying real-valued matrix M is first quantized to generate one-bit
observations and then a subset of positive entries is revealed. Under the
assumption that M has bounded nuclear norm, we provide recovery guarantees for
two different observation models: 1) M parameterizes a distribution that
generates a binary matrix, 2) M is thresholded to obtain a binary matrix. For
the first case, we propose a "shifted matrix completion" method that recovers M
using only a subset of indices corresponding to ones, while for the second
case, we propose a "biased matrix completion" method that recovers the
(thresholded) binary matrix. Both methods yield strong error bounds --- if M is
n by n, the Frobenius error is bounded as O(1/((1-rho)n), where 1-rho denotes
the fraction of ones observed. This implies a sample complexity of O(n\log n)
ones to achieve a small error, when M is dense and n is large. We extend our
methods and guarantees to the inductive matrix completion problem, where rows
and columns of M have associated features. We provide efficient and scalable
optimization procedures for both the methods and demonstrate the effectiveness
of the proposed methods for link prediction (on real-world networks consisting
of over 2 million nodes and 90 million links) and semi-supervised clustering
tasks
Analysis of a Collaborative Filter Based on Popularity Amongst Neighbors
In this paper, we analyze a collaborative filter that answers the simple
question: What is popular amongst your friends? While this basic principle
seems to be prevalent in many practical implementations, there does not appear
to be much theoretical analysis of its performance. In this paper, we partly
fill this gap. While recent works on this topic, such as the low-rank matrix
completion literature, consider the probability of error in recovering the
entire rating matrix, we consider probability of an error in an individual
recommendation (bit error rate (BER)). For a mathematical model introduced in
[1],[2], we identify three regimes of operation for our algorithm (named
Popularity Amongst Friends (PAF)) in the limit as the matrix size grows to
infinity. In a regime characterized by large number of samples and small
degrees of freedom (defined precisely for the model in the paper), the
asymptotic BER is zero; in a regime characterized by large number of samples
and large degrees of freedom, the asymptotic BER is bounded away from 0 and 1/2
(and is identified exactly except for a special case); and in a regime
characterized by a small number of samples, the algorithm fails. We also
present numerical results for the MovieLens and Netflix datasets. We discuss
the empirical performance in light of our theoretical results and compare with
an approach based on low-rank matrix completion.Comment: 47 pages. Submitted to IEEE Transactions on Information Theory
(revised in July 2011). A shorter version would be presented at ISIT 201
- β¦