627 research outputs found

    Ensemble divide and conquer approach to solve the rating scores’ deviation in recommendation system

    Get PDF
    The rating matrix of a personalized recommendation system contains a high percentage of unknown rating scores which lowers the quality of the prediction. Besides, during data streaming into memory, some rating scores are misplaced from its appropriate cell in the rating matrix which also decrease the quality of the prediction. The singular value decomposition algorithm predicts the unknown rating scores based on the relation between the implicit feedback of both users and items, but exploiting neither the user similarity nor item similarity which leads to low accuracy predictions. There are several factorization methods used in improving the prediction performance of the collaborative filtering technique such as baseline, matrix factorization, neighbour-base. However, the prediction performance of the collaborative filtering using factorization methods is still low while baseline and neighbours-base have limitations in terms of over fitting. Therefore, this paper proposes Ensemble Divide and Conquer (EDC) approach for solving 2 main problems which are the data sparsity and the rating scores’ deviation (misplace). The EDC approach is founded by the Singular Value Decomposition (SVD) algorithm which extracts the relationship between the latent feedback of users and the latent feedback of the items. Furthermore, this paper addresses the scale of rating scores as a sub problem which effect on the rank approximation among the users’ features. The latent feedback of the users and items are also SVD factors. The results using the EDC approach are more accurate than collaborative filtering and existing methods of matrix factorization namely SVD, baseline, matrix factorization and neighbours-base. This indicates the significance of the latent feedback of both users and items against the different factorization features in improving the prediction accuracy of the collaborative filtering technique

    Deep Item-based Collaborative Filtering for Top-N Recommendation

    Full text link
    Item-based Collaborative Filtering(short for ICF) has been widely adopted in recommender systems in industry, owing to its strength in user interest modeling and ease in online personalization. By constructing a user's profile with the items that the user has consumed, ICF recommends items that are similar to the user's profile. With the prevalence of machine learning in recent years, significant processes have been made for ICF by learning item similarity (or representation) from data. Nevertheless, we argue that most existing works have only considered linear and shallow relationship between items, which are insufficient to capture the complicated decision-making process of users. In this work, we propose a more expressive ICF solution by accounting for the nonlinear and higher-order relationship among items. Going beyond modeling only the second-order interaction (e.g. similarity) between two items, we additionally consider the interaction among all interacted item pairs by using nonlinear neural networks. Through this way, we can effectively model the higher-order relationship among items, capturing more complicated effects in user decision-making. For example, it can differentiate which historical itemsets in a user's profile are more important in affecting the user to make a purchase decision on an item. We treat this solution as a deep variant of ICF, thus term it as DeepICF. To justify our proposal, we perform empirical studies on two public datasets from MovieLens and Pinterest. Extensive experiments verify the highly positive effect of higher-order item interaction modeling with nonlinear neural networks. Moreover, we demonstrate that by more fine-grained second-order interaction modeling with attention network, the performance of our DeepICF method can be further improved.Comment: 25 pages, submitted to TOI

    Mining Web Dynamics for Search

    Get PDF
    Billions of web users collectively contribute to a dynamic web that preserves how information sources and descriptions change over time. This dynamic process sheds light on the quality of web content, and even indicates the temporal properties of information needs expressed via queries. However, existing commercial search engines typically utilize one crawl of web content (the latest) without considering the complementary information concealed in web dynamics. As a result, the generated rankings may be biased due to the efficiency of knowledge on page or hyperlink evolution, and the time-sensitive facet within search quality, e.g., freshness, has to be neglected. While previous research efforts have been focused on exploring the temporal dimension in retrieval process, few of them showed consistent improvements on large-scale real-world archival web corpus with a broad time span.We investigate how to utilize the changes of web pages and hyperlinks to improve search quality, in terms of freshness and relevance of search results. Three applications that I have focused on are: (1) document representation, in which the anchortext (short descriptive text associated with hyperlinks) importance is estimated by considering its historical status; (2) web authority estimation, in which web freshness is quantified and utilized for controlling the authority propagation; and (3) learning to rank, in which freshness and relevance are optimized simultaneously in an adaptive way depending on query type. The contributions of this thesis are: (1) incorporate web dynamics information into critical components within search infrastructure in a principled way; and (2) empirically verify the proposed methods by conducting experiments based on (or depending on) a large-scale real-world archival web corpus, and demonstrated their superiority over existing state-of-the-art

    Social informatics

    Get PDF
    5th International Conference, SocInfo 2013, Kyoto, Japan, November 25-27, 2013, Proceedings</p

    Exploiting Latent Information in Recommender Systems

    No full text
    This thesis exploits latent information in personalised recommendation, and investigates how this information can be used to improve recommender systems. The investigations span three directions: scalar rating-based collaborative filtering, distributional rating-based collaborative filtering, and distributional ratingbased hybrid filtering. In the first investigation, the thesis discovers through data analysis three problems in nearest neighbour collaborative filtering — item irrelevance, preference imbalance, and biased average — and identifies a solution: incorporating “target awareness” in the computation of user similarity and rating deviation. Two new algorithms are subsequently proposed. Quantitative experiments show that the new algorithms, especially the first one, are able to significantly improve the performance under normal situations. They do not however excel in cold-start situations due to greater demand of data. The second investigation builds upon the experimental analysis of the first investigation, and examines the use of discrete probabilistic distributional modelling throughout the recommendation process. It encompasses four ideas: 1) distributional input rating, which enables the explicit representation of noise patterns in user inputs; 2) distributional voting profile, which enables the preservation of not only shift but also spread and peaks in user’s rating habits; 3) distributional similarity, which enables the untangled and separated similarity computation of the likes and the dislikes; and 4) distributional prediction, which enables the communication of the uncertainty, granularity, and ambivalence in the recommendation results. Quantitative experiments show that this model is able to improve the effectiveness of recommendation compared to the scalar model and other published discrete probabilistic models, especially in terms of binary and list recommendation accuracy. The third investigation is based on an analysis regarding the relationship between rating, item content, item quality, and “intangibles”, and is enabled by the discrete probabilistic model proposed in the second investigation. Based on the analysis, a fundamentally different hybrid filtering structure is proposed, where the hybridisation strategy is neither linear nor sequential, but of a divide-and-conquer shape backed by probabilistic derivation. Experimental results show that it is able to outperform the standard linear and sequential hybridisation structures

    March madness prediction using machine learning techniques

    Get PDF
    Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceMarch Madness describes the final tournament of the college basketball championship, considered by many as the biggest sporting event in the United States - moving every year tons of dollars in both bets and television. Besides that, there are 60 million Americans who fill out their tournament bracket every year, and anything is more likely than hit all 68 games. After collecting and transforming data from Sports-Reference.com, the experimental part consists of preprocess the data, evaluate the features to consider in the models and train the data. In this study, based on tournament data over the last 20 years, Machine Learning algorithms like Decision Trees Classifier, K-Nearest Neighbors Classifier, Stochastic Gradient Descent Classifier and others were applied to measure the accuracy of the predictions and to be compared with some benchmarks. Despite of the most important variables seemed to be those related to seeds, shooting and the number of participations in the tournament, it was not possible to define exactly which ones should be used in the modeling and all ended up being used. Regarding the results, when training the entire dataset, the accuracy ranges from 65 to 70%, where Support Vector Classification yields the best results. When compared with picking the highest seed, these results are slightly lower. On the other hand, when predicting the Tournament of 2017, the Support Vector Classification and the Multi-Layer Perceptron Classifier reach 85 and 79% of accuracy, respectively. In this sense, they surpass the previous benchmark and the most respected websites and statistics in the field. Given some existing constraints, it is quite possible that these results could be improved and deepened in other ways. Meanwhile, this project can be referenced and serve as a basis for the future work

    Dynamic Estimation of Rater Reliability using Multi-Armed Bandits

    Get PDF
    One of the critical success factors for supervised machine learning is the quality of target values, or predictions, associated with training instances. Predictions can be discrete labels (such as a binary variable specifying whether a blog post is positive or negative) or continuous ratings (for instance, how boring a video is on a 10-point scale). In some areas, predictions are readily available, while in others, the eort of human workers has to be involved. For instance, in the task of emotion recognition from speech, a large corpus of speech recordings is usually available, and humans denote which emotions are present in which recordings
    corecore