288 research outputs found
Recommender Systems in Light of Big Data
The growth in the usage of the web, especially e-commerce website, has led to the development of recommender system (RS) which aims in personalizing the web content for each user and reducing the cognitive load of information on the user. However, as the world enters Big Data era and lives through the contemporary data explosion, the main goal of a RS becomes to provide millions of high quality recommendations in few seconds for the increasing number of users and items. One of the successful techniques of RSs is collaborative filtering (CF) which makes recommendations for users based on what other like-mind users had preferred. Despite its success, CF is facing some challenges posed by Big Data, such as: scalability, sparsity and cold start. As a consequence, new approaches of CF that overcome the existing problems have been studied such as Singular value decomposition (SVD). This paper surveys the literature of RSs and reviews the current state of RSs with the main concerns surrounding them due to Big Data. Furthermore, it investigates thoroughly SVD, one of the promising approaches expected to perform well in tackling Big Data challenges, and provides an implementation to it using some of the successful Big Data tools (i.e. Apache Hadoop and Spark). This implementation is intended to validate the applicability of, existing contributions to the field of, SVD-based RSs as well as validated the effectiveness of Hadoop and spark in developing large-scale systems. The implementation has been evaluated empirically by measuring mean absolute error which gave comparable results with other experiments conducted, previously by other researchers, on a relatively smaller data set and non-distributed environment. This proved the scalability of SVD-based RS and its applicability to Big Data
Fast Updating Truncated SVD for Representation Learning with Sparse Matrices
Updating a truncated Singular Value Decomposition (SVD) is crucial in
representation learning, especially when dealing with large-scale data matrices
that continuously evolve in practical scenarios. Aligning SVD-based models with
fast-paced updates becomes increasingly important. Existing methods for
updating truncated SVDs employ Rayleigh-Ritz projection procedures, where
projection matrices are augmented based on original singular vectors. However,
these methods suffer from inefficiency due to the densification of the update
matrix and the application of the projection to all singular vectors. To
address these limitations, we introduce a novel method for dynamically
approximating the truncated SVD of a sparse and temporally evolving matrix. Our
approach leverages sparsity in the orthogonalization process of augmented
matrices and utilizes an extended decomposition to independently store
projections in the column space of singular vectors. Numerical experiments
demonstrate a remarkable efficiency improvement of an order of magnitude
compared to previous methods. Remarkably, this improvement is achieved while
maintaining a comparable precision to existing approaches
- …