Skip to main content
Article thumbnail
Location of Repository

Experimentation, Algorithms

By Yew Jin Lim and Yee Whye Teh

Abstract

Singular value decomposition (SVD) is a matrix decomposition algorithm that returns the optimal (in the sense of squared error) low-rank decomposition of a matrix. SVD has found widespread use across a variety of machine learning applications, where its output is interpreted as compact and informative representations of data. The Netflix Prize challenge, and collaborative filtering in general, is an ideal application for SVD, since the data is a matrix of ratings given by users to movies. It is thus not surprising to observe that most currently successful teams use SVD, either with an extension, or to interpolate with results returned by other algorithms. Unfortunately SVD can easily overfit due to the extreme data sparsity of the matrix in the Netflix Prize challenge, and care must be taken to regularize properly. In this paper, we propose a Bayesian approach to alleviate overfitting in SVD, where priors are introduced and all parameters are integrated out using variational inference. We show experimentally that this gives significantly improved results over vanilla SVD. For truncated SVDs of rank 5, 10, 20, and 30, our proposed Bayesian approach achieves 2.2% improvement over a naïve approach, 1.6 % improvement over a gradient descent approach dealing with unobserved entries properly, and 0.9 % improvement over a maximum a posteriori (MAP) approach. Categories and Subject Descriptors I.2.6 [Machine Learning]: Engineering applications—applications of technique

Topics: machine learning, SVD, Variational Inference
Year: 2009
OAI identifier: oai:CiteSeerX.psu:10.1.1.135.7961
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://citeseerx.ist.psu.edu/v... (external link)
  • http://www.cs.berkeley.edu/~yw... (external link)
  • Suggested articles


    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.