Context-aware user modeling strategies for journey plan recommendation

Popular journey planning systems, like Google Maps or Yahoo! Maps, usually ignore user’s preferences and context. This paper shows how we applied context-aware recommendation technologies in an existing journey planning mobile application to provide personalized and context-dependent recommendations to users. We describe two different strategies for context-aware user modeling in the journey planning domain. We present an extensive performance comparison of the proposed strategies by conducting a user-centric study in addition to a traditional offline evaluation methodPeer ReviewedPostprint (published version

Deliverable D.8.4. Social Data Visualization and Navigation Services:3rd Year Update

Within the Open Discovery Space our study (T.8.4) focused on ”Enhanced Social Data Visualization & Navigation Services. This deliverable provides the prototype report regarding the deployment of adapted visualization and navigation services to be integrated in the ODS Social Data Management Layer.Project co-funded by the European Commission within the ICT Policy Support Programme, CIP Competitiveness and innovation framework programme 2007 - 2013. Grant agreement no: 29722

User-centric evaluation of recommender systems in social learning platforms: Accuracy is just the tip of the iceberg

Recommender systems provide users with content they might be interested in. Conventionally, recommender systems are evaluated mostly by using prediction accuracy metrics only. But the ultimate goal of a recommender system is to increase user satisfaction. Therefore, evaluations that measure user satisfaction should also be performed before deploying a recommender system in a real target environment. Such evaluations are laborious and complicated compared to the traditional, data-centric evaluations, though. In this study, we carried out a user-centric evaluation of state-of-the-art recommender systems as well as a graph-based approach in the ecologically valid setting of an authentic social learning platform. We also conducted a data-centric evaluation on the same data to investigate the added value of user-centric evaluations and how user satisfaction of a recommender system is related to its performance in terms of accuracy metrics. Our findings suggest that user-centric evaluation results are not necessarily in line with data-centric evaluation results. We conclude that the traditional evaluation of recommender systems in terms of prediction accuracy only does not suffice to judge performance of recommender systems on the user side. Moreover, the user-centric evaluation provides valuable insights in how candidate algorithms perform on each of the five quality metrics for recommendations: usefulness, accuracy, novelty, diversity, and serendipity

Study of individual users and groups : perceptions of recommender systems performance

[SPA]La cantidad de información disponible en Internet ha crecido enormemente desde la aparición de nuevas tecnologías y redes sociales. Como consecuencia de este crecimiento, ha nacido un problema denominado ‘sobrecarga de información’. La solución a dicho problema es el uso de sistemas de recomendación, de esta manera conseguimos filtrar aquella información que realmente es relevante para el usuario, se aligera el tiempo empleado para la toma de decisiones a la hora de realizar una compra, seleccionar un libro para leer o una película para ver. Sin embargo, ¿cómo podemos estar seguros de que estamos usando el mejor sistema de recomendación? Muchos investigadores han analizado el empleo de nuevas medidas subjetivas para medir la percepción que los usuarios tienen del sistema, ya que la satisfacción de los usuarios asegura la optimización del sistema. Con el fin de encontrar la relación entre dichas medidas y la calidad del sistema, realizamos un estudio con usuarios reales en el dominio de las películas, con el propósito de analizar cómo afectan estas medidas a su satisfacción. En este estudio se han examinado 6 algoritmos de recomendación diferentes (tres algoritmos de filtrado colaborativo, un algoritmo híbrido y dos algoritmos básicos) a través de una evaluación offline con el fin de identificar los mejores parámetros para configurar cada uno de los algoritmos, seguida por una evaluación online con usuarios reales. En dicha evaluación online, los usuarios tienen que comparar seis listas con películas recomendadas por cada uno de los algoritmos, además de responder a una serie de preguntas para medir la Precisión, Novedad, Confianza, Variedad, Efectividad y Calidad de cada uno de los algoritmos de recomendación empleados. Además, también llevamos a cabo el análisis de recomendaciones grupales con el propósito de demostrar que no se necesitan sistemas complejos para crear buenas recomendaciones para grupos. También investigamos si las medidas subjetivas mencionadas con anterioridad influyen en la satisfacción del grupo con las recomendaciones recibidas.[ENG]The amount of data available on the Internet has enormously increased since the apparition of new technologies and social networks. As a consequence, a problem has emerged called ‘information overload’. The solution for this problem is the use of recommender systems. However, how can we be sure that we are using the best system to make recommendations? Lots of researchers [1] [4] [5] [8] have discussed the use of new subjective metrics to measure the perception of the system that users have about it since users satisfaction ensures the goodness of the recommender. To figure out the relation among these metrics and the quality of a system, we offer a user study in the movie domain with the aim of analyzing how these metrics affect their satisfaction. This paper examines six different algorithms (three common collaborative filtering, one hybrid, and two basics) through an offline evaluation to identify the best parameter for each of them, followed by the online evaluation with real users. In this online experiment, users have to compare six lists of recommendations produced by each algorithm regarding the measurements of Accuracy, Novelty, Understands Me, Diversity, Effectiveness and Quality.Our study also covers the analysis of group recommendations with the purpose of proving that there is no need for complex systems in order to make good group recommendations. Moreover, we investigate whether the subjective metrics above mentioned influence in group satisfaction.Escuela Técnica Superior de Ingeniería de TelecomunicaciónUniversidad Politécnica de Cartagen

Customers’ loyalty model in the design of e-commerce recommender systems

Recommender systems have been adopted in most modern online platforms to guide users in finding more suitable items that match their interests. Previous studies showed that recommender systems impact the buying behavior of e-commerce customers. However, service providers are more concerned about the continuing behavior of their customers, specifically customers’ loyalty, which is an important factor to increase service providers’ share of wallet. Therefore, this study aimed to investigate the customers’ loyalty factors in online shopping towards e-commerce recommender systems. To address the research objectives, a new research model was proposed based on the Cognition-Affect-Behavior model. To validate the research model, a quantitative methodology was utilized to gather the relevant data. Using a survey method, a total of 310 responses were gathered to examine the impacts of the identified factors on customers’ loyalty towards Amazon’s recommender system. Data was analysed using Partial Least Square Structural Equation Modelling. The results of the analysis indicated that Usability (P=0.467, t=5.139, p<0.001), Service Interaction (P=0.304, t=4.42, p<0.001), Website Quality (P=0.625, t=15.304, p<0.001), Accuracy (P=0.397, t=6.144, p<0.001), Novelty (P=0.289, t=4.406, p<0.001), Diversity (P=0.142, t=2.503, p<0.001), Recommendation Quality (P=0.423, t=7.719, p<0.001), Explanation (P=0.629, t=15.408, p<0.001), Transparency (P=0.279, t=5.859, p<0.001), Satisfaction (P=0.152, t=3.045, p<0.001) and Trust (P=0.706, t=14.14, p<0.001) have significant impacts on customers’ loyalty towards the recommender systems in online shopping. Information quality, however, did not affect the quality of the website that hosted the recommender system. The findings demonstrated that accuracy-oriented measures were insufficient in understanding customer behavior, and other quality factors, such as diversity, novelty, and transparency could improve customers’ loyalty towards recommender systems. The outcomes of the study indicated the significant impact of the website quality on customers’ loyalty. The developed model would be practical in helping the service providers in understanding the impacts of the identified factors in the proposed customers’ loyalty model. The outcomes of the study could also be used in the design of recommender systems and the deployed algorithm

Evaluating Recommender Systems: Survey and Framework

The comprehensive evaluation of the performance of a recommender system is a complex endeavor: many facets need to be considered in configuring an adequate and effective evaluation setting. Such facets include, for instance, defining the specific goals of the evaluation, choosing an evaluation method, underlying data, and suitable evaluation metrics. In this paper, we consolidate and systematically organize this dispersed knowledge on recommender systems evaluation. We introduce the “Framework for EValuating Recommender systems” (FEVR) that we derive from the discourse on recommender systems evaluation. In FEVR, we categorize the evaluation space of recommender systems evaluation. We postulate that the comprehensive evaluation of a recommender system frequently requires considering multiple facets and perspectives in the evaluation. The FEVR framework provides a structured foundation to adopt adequate evaluation configurations that encompass this required multi-facettedness and provides the basis to advance in the field. We outline and discuss the challenges of a comprehensive evaluation of recommender systems, and provide an outlook on what we need to embrace and do to move forward as a research community