22 research outputs found
Noisy low-rank matrix completion with general sampling distribution
In the present paper, we consider the problem of matrix completion with
noise. Unlike previous works, we consider quite general sampling distribution
and we do not need to know or to estimate the variance of the noise. Two new
nuclear-norm penalized estimators are proposed, one of them of "square-root"
type. We analyse their performance under high-dimensional scaling and provide
non-asymptotic bounds on the Frobenius norm error. Up to a logarithmic factor,
these performance guarantees are minimax optimal in a number of circumstances.Comment: Published in at http://dx.doi.org/10.3150/12-BEJ486 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Online Learning with Low Rank Experts
We consider the problem of prediction with expert advice when the losses of
the experts have low-dimensional structure: they are restricted to an unknown
-dimensional subspace. We devise algorithms with regret bounds that are
independent of the number of experts and depend only on the rank . For the
stochastic model we show a tight bound of , and extend it to
a setting of an approximate subspace. For the adversarial model we show an
upper bound of and a lower bound of
Learning with the Weighted Trace-norm under Arbitrary Sampling Distributions
We provide rigorous guarantees on learning with the weighted trace-norm under
arbitrary sampling distributions. We show that the standard weighted trace-norm
might fail when the sampling distribution is not a product distribution (i.e.
when row and column indexes are not selected independently), present a
corrected variant for which we establish strong learning guarantees, and
demonstrate that it works better in practice. We provide guarantees when
weighting by either the true or empirical sampling distribution, and suggest
that even if the true distribution is known (or is uniform), weighting by the
empirical distribution may be beneficial
Convex Tensor Decomposition via Structured Schatten Norm Regularization
We discuss structured Schatten norms for tensor decomposition that includes
two recently proposed norms ("overlapped" and "latent") for
convex-optimization-based tensor decomposition, and connect tensor
decomposition with wider literature on structured sparsity. Based on the
properties of the structured Schatten norms, we mathematically analyze the
performance of "latent" approach for tensor decomposition, which was
empirically found to perform better than the "overlapped" approach in some
settings. We show theoretically that this is indeed the case. In particular,
when the unknown true tensor is low-rank in a specific mode, this approach
performs as good as knowing the mode with the smallest rank. Along the way, we
show a novel duality result for structures Schatten norms, establish the
consistency, and discuss the identifiability of this approach. We confirm
through numerical simulations that our theoretical prediction can precisely
predict the scaling behavior of the mean squared error.Comment: 12 pages, 3 figure
Generalization error bounds for kernel matrix completion and extrapolation
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Prior information can be incorporated in matrix completion to improve estimation accuracy and extrapolate the missing entries. Reproducing kernel Hilbert spaces provide tools to leverage the said prior information, and derive more reliable algorithms. This paper analyzes the generalization error of such approaches, and presents numerical tests confirming the theoretical resultsThis work is supported by ERDF funds (TEC2013-41315-R and TEC2016-75067-C4-2), the Catalan Government (2017 SGR 578), and NSF grants(1500713, 1514056, 1711471 and 1509040).Peer ReviewedPostprint (published version
A Max-Norm Constrained Minimization Approach to 1-Bit Matrix Completion
We consider in this paper the problem of noisy 1-bit matrix completion under
a general non-uniform sampling distribution using the max-norm as a convex
relaxation for the rank. A max-norm constrained maximum likelihood estimate is
introduced and studied. The rate of convergence for the estimate is obtained.
Information-theoretical methods are used to establish a minimax lower bound
under the general sampling model. The minimax upper and lower bounds together
yield the optimal rate of convergence for the Frobenius norm loss.
Computational algorithms and numerical performance are also discussed.Comment: 33 pages, 3 figure
Matrix Completion via Max-Norm Constrained Optimization
Matrix completion has been well studied under the uniform sampling model and
the trace-norm regularized methods perform well both theoretically and
numerically in such a setting. However, the uniform sampling model is
unrealistic for a range of applications and the standard trace-norm relaxation
can behave very poorly when the underlying sampling scheme is non-uniform.
In this paper we propose and analyze a max-norm constrained empirical risk
minimization method for noisy matrix completion under a general sampling model.
The optimal rate of convergence is established under the Frobenius norm loss in
the context of approximately low-rank matrix reconstruction. It is shown that
the max-norm constrained method is minimax rate-optimal and yields a unified
and robust approximate recovery guarantee, with respect to the sampling
distributions. The computational effectiveness of this method is also
discussed, based on first-order algorithms for solving convex optimizations
involving max-norm regularization.Comment: 33 page