Rank and relevance in novelty and diversity metrics for recommender systems

This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in RecSys '11 Proceedings of the fifth ACM conference on Recommender systems, http://dx.doi.org/10.1145/2043932.2043955The Recommender Systems community is paying increasing attention to novelty and diversity as key qualities beyond accuracy in real recommendation scenarios. Despite the raise of interest and work on the topic in recent years, we find that a clear common methodological and conceptual ground for the evaluation of these dimensions is still to be consolidated. Different evaluation metrics have been reported in the literature but the precise relation, distinction or equivalence between them has not been explicitly studied. Furthermore, the metrics reported so far miss important properties such as taking into consideration the ranking of recommended items, or whether items are relevant or not, when assessing the novelty and diversity of recommendations. We present a formal framework for the definition of novelty and diversity metrics that unifies and generalizes several state of the art metrics. We identify three essential ground concepts at the roots of novelty and diversity: choice, discovery and relevance, upon which the framework is built. Item rank and relevance are introduced through a probabilistic recommendation browsing model, building upon the same three basic concepts. Based on the combination of ground elements, and the assumptions of the browsing model, different metrics and variants unfold. We report experimental observations which validate and illustrate the properties of the proposed metrics.This work is supported by the Spanish Government (TIN2011- 28538-C02-01), and the Government of Madrid (S2009TIC-1542)

Beyond Personalization: Research Directions in Multistakeholder Recommendation

Recommender systems are personalized information access applications; they are ubiquitous in today's online environment, and effective at finding items that meet user needs and tastes. As the reach of recommender systems has extended, it has become apparent that the single-minded focus on the user common to academic research has obscured other important aspects of recommendation outcomes. Properties such as fairness, balance, profitability, and reciprocity are not captured by typical metrics for recommender system evaluation. The concept of multistakeholder recommendation has emerged as a unifying framework for describing and understanding recommendation settings where the end user is not the sole focus. This article describes the origins of multistakeholder recommendation, and the landscape of system designs. It provides illustrative examples of current research, as well as outlining open questions and research directions for the field.Comment: 64 page

Collaborative-demographic hybrid for financial: product recommendation

Internship Report presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsDue to the increased availability of mature data mining and analysis technologies supporting CRM processes, several financial institutions are striving to leverage customer data and integrate insights regarding customer behaviour, needs, and preferences into their marketing approach. As decision support systems assisting marketing and commercial efforts, Recommender Systems applied to the financial domain have been gaining increased attention. This thesis studies a Collaborative- Demographic Hybrid Recommendation System, applied to the financial services sector, based on real data provided by a Portuguese private commercial bank. This work establishes a framework to support account managers’ advice on which financial product is most suitable for each of the bank’s corporate clients. The recommendation problem is further developed by conducting a performance comparison for both multi-output regression and multiclass classification prediction approaches. Experimental results indicate that multiclass architectures are better suited for the prediction task, outperforming alternative multi-output regression models on the evaluation metrics considered. Withal, multiclass Feed-Forward Neural Networks, combined with Recursive Feature Elimination, is identified as the topperforming algorithm, yielding a 10-fold cross-validated F1 Measure of 83.16%, and achieving corresponding values of Precision and Recall of 84.34%, and 85.29%, respectively. Overall, this study provides important contributions for positioning the bank’s commercial efforts around customers’ future requirements. By allowing for a better understanding of customers’ needs and preferences, the proposed Recommender allows for more personalized and targeted marketing contacts, leading to higher conversion rates, corporate profitability, and customer satisfaction and loyalty

IFIP TC 13 Seminar: trends in HCI proceedings, March 26, 2007, Salamanca (Spain)

Actas del 13o. Seminario de la International Federation for Information Processing (IFIP), celebrado en Salamanca el 26 de marzo de 2007, sobre las nuevas líneas de investigación en la interacción hombre-máquina, gestión del conocimiento y enseñanza por la Web

Tackling the problems of diversity in recommender systems

Master of ScienceDepartment of Computing and Information SciencesWilliam H. HsuA recommender system is a computational mechanism for information filtering, where users provide recommendations (in the form of ratings or selecting items) as inputs, which the system then aggregates and directs to appropriate recipients. With the advent of web based media and publicity methods, the age where standardized methods of publicity, sales, production and marketing strategies do not. As such, in many markets the users are given a wide range of products and information to choose which product they like, to find a way out of this recommender systems are used in a way similar to the live social scenario, that is a user tries to get reviews from friends before opting for a product in a similar way recommender system tries to be a friend who recommends the options. Most of the recommender systems currently developed solely accuracy driven, i.e., reducing the Mean Absolute Error (MAE) between the predictions of the recommender system and actual ratings of the user. This leads to various problems for recommender systems such as lack of diversity and freshness. Lack of diversity arises when the recommender system is overly focused on accuracy by recommending a set of items, in which all of the items are too similar to each other, because they are predicted to be liked by the user. Lack of freshness also arises with overly focusing on accuracy but as a limitation on the set of items recommended making it overly predictable. This thesis work is directed at addressing the issues of diversity, by developing an approach, where a threshold of accuracy (in terms of Mean Absolute Error in prediction) is maintained while trying to diversify the set of item recommendations. Here for the problem of diversity a combination of Attribute-based diversification and user preference based diversification is done. This approach is then evaluated using non-classical methods along with evaluating the base recommender algorithm to prove that diversification is indeed is possible with a mixture of collaborative and content based approach

Controlling the Fairness / Accuracy Tradeoff in Recommender Systems

Recommender systems are one of the most pervasive applications of machine learning. They play a pivotal role in helping users find items tailored to their taste. Although these systems intend to assist people in their information needs, they can cause implicit or explicit discrimination against individuals or groups. There are several ways that different biases can creep into recommender systems. Reflection of societal and historical prejudices in datasets and during the data collection process, lack of sufficient data on minority groups, lack of suitable evaluation methods and model designs to detect these biases and lessen the unfairness caused by them are among the many reasons for unfairness in these systems. A system needs to defend against the biases in recommendation output to prevent harm and unfairness. However, integrating the goal of fairness with accuracy in recommender systems is challenging, primarily because of this goal's significant trade-offs with accuracy. Accuracy in recommender systems is the ability of that system to predict users' needs and interests accurately. On the other hand, fairness is a complicated concept with a variety of definitions. To use fairness as an objective, we need to define it based on the application area and the context of a problem. Additionally, we need to specify the fairness concerns of the different stakeholders involved in the recommender systems and the fairness priorities of a system. Any of these aspects might disagree with the goal of accuracy. For example, if fairness for content providers is more exposure to users, increasing it might cause a reduction in accuracy. Therefore, controlling the trade-off between accuracy and fairness becomes essential. Throughout this dissertation, several recommendation models and re-ranking approaches are presented that aim to address this problem using in- and post- processing methods. These approaches show promising results, but it is worth mentioning that they have intrinsic limitations and, therefore, shouldn't be considered ultimate solutions

Recommender system performance evaluation and prediction: information retrieval perspective

Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, octubre de 201

Architecture Supporting Computational Trust Formation

Trust is a concept that has been used in computing to support better decision making. For example, trust can be used in access control. Trust can also be used to support service selection. Although certain elements of trust such as reputation has gained widespread acceptance, a general model of trust has so far not seen widespread usage. This is due to the challenges of implementing a general trust model. In this thesis, a middleware based approach is proposed to address the implementation challenges. The thesis proposes a general trust model known as computational trust. Computational trust is based on research in social psychology. An individual’s computational trust is formed with the support of the proposed computational trust architecture. The architecture consists of a middleware and middleware clients. The middleware can be viewed as a representation of the individual that shares its knowledge with all the middleware clients. Each application uses its own middleware client to form computational trust for its decision making needs. Computational trust formation can be adapted to changing circumstances. The thesis also proposed algorithms for computational trust formation. Experiments, evaluations and scenarios are also presented to demonstrate the feasibility of the middleware based approach to computational trust formation