Quantitative analysis of Matthew effect and sparsity problem of recommender systems

Recommender systems have received great commercial success. Recommendation has been used widely in areas such as e-commerce, online music FM, online news portal, etc. However, several problems related to input data structure pose serious challenge to recommender system performance. Two of these problems are Matthew effect and sparsity problem. Matthew effect heavily skews recommender system output towards popular items. Data sparsity problem directly affects the coverage of recommendation result. Collaborative filtering is a simple benchmark ubiquitously adopted in the industry as the baseline for recommender system design. Understanding the underlying mechanism of collaborative filtering is crucial for further optimization. In this paper, we do a thorough quantitative analysis on Matthew effect and sparsity problem in the particular context setting of collaborative filtering. We compare the underlying mechanism of user-based and item-based collaborative filtering and give insight to industrial recommender system builders

ROBUST LOW-RANK MATRIX FACTORIZATION WITH MISSING DATA BY MINIMIZING L1 LOSS APPLIED TO COLLABORATIVE FILTERING

In this age of information overload and plethora of choices, people increasingly rely on automatic recommender systems to tell them what suits their needs. A very effective approach for creating recommender systems is collaborative filtering, which is the task of predicting the preference/rating that a user would assign to an item based on preference data of that user and preference data of other users. One way to conduct collaborative filtering is through dimensionality reduction. The underlying concept of the approach lies in the belief that there are only a few features (reduced dimensions) that influence the user’s choice. In this paper we use low rank matrix factorization for dimensionality reduction. Singular Value Decomposition (SVD), which is minimizing the L2 norm is the most popular technique to perform matrix factorization. However, in most recommendation system data sets, often the users only rate a small amount of items, which creates missing data. As a result SVD fails. In recent years L1 norm has gained much importance and popularity because it is robust to outliers and missing data. In this thesis we use alternate convex optimization to perform L1 norm minimization to solve the matrix factorization problem and apply it to collaborative filtering. We also review some of the major challenges that collaborative filtering faces today and some of the other techniques used. Additionally, this thesis discusses the importance and future of collaborative filtering in medical applications that concerns the database of patient history (prescriptions/symptoms) and how it can be used as a predictive task for the future of the patient

Impact of Matrix Factorization and Regularization Hyperparameter on a Recommender System for Movies

Recommendation system is developed to match consumers with product to meet their variety of special needs and tastes in order to enhance user satisfaction and loyalty. The popularity of personalized recommendation system has been increased in recent years and applied in several areas include movies, songs, books, news, friend recommendations on social media, travel products, and other products in general. Collaborative Filtering methods are widely used in recommendation systems. The collaborative filtering method is divided into neighborhood-based and model-based. In this study, we are implementing matrix factorization which is part of model-based that learns latent factor for each user and item and uses them to make rating predictions. The method will be trained using stochastic gradient descent with additional tricks and optimization of regularization hyperparameter. In the end, neighborhood-based collaborative filtering and matrix factorization with different values of regularization hyperparameter will be compared. Our result shows that matrix factorization method with lowest regularization hyperparameter outperformed the other methods in term of RMSE score. In this study, the used functions are available from Graphlab and using Movielens 100k data set for building the recommendation systems

Recommendation Systems Based on Association Rule Mining for a Target Object by Evolutionary Algorithms

Recommender systems are designed for offering products to the potential customers. Collaborative Filtering is known as a common way in Recommender systems which offers recommendations made by similar users in the case of entering time and previous transactions. Low accuracy of suggestions due to a database is one of the main concerns about collaborative filtering recommender systems. In this field, numerous researches have been done using associative rules for recommendation systems to improve accuracy but runtime of rule-based recommendation systems is high and cannot be used in the real world. So, many researchers suggest using evolutionary algorithms for finding relative best rules at runtime very fast. The present study investigated the works done for producing associative rules with higher speed and quality. In the first step Apriori-based algorithm will be introduced which is used for recommendation systems and then the Particle Swarm Optimization algorithm will be described and the issues of these 2 work will be discussed. Studying this research could help to know the issues in this research field and produce suggestions which have higher speed and quality