Sustainable Transparency in Recommender Systems: Bayesian Ranking of Images for Explainability

Recommender Systems have become crucial in the modern world, commonly guiding users towards relevant content or products, and having a large influence over the decisions of users and citizens. However, ensuring transparency and user trust in these systems remains a challenge; personalized explanations have emerged as a solution, offering justifications for recommendations. Among the existing approaches for generating personalized explanations, using visual content created by the users is one particularly promising option, showing a potential to maximize transparency and user trust. Existing models for explaining recommendations in this context face limitations: sustainability has been a critical concern, as they often require substantial computational resources, leading to significant carbon emissions comparable to the Recommender Systems where they would be integrated. Moreover, most models employ surrogate learning goals that do not align with the objective of ranking the most effective personalized explanations for a given recommendation, leading to a suboptimal learning process and larger model sizes. To address these limitations, we present BRIE, a novel model designed to tackle the existing challenges by adopting a more adequate learning goal based on Bayesian Pairwise Ranking, enabling it to achieve consistently superior performance than state-of-the-art models in six real-world datasets, while exhibiting remarkable efficiency, emitting up to 75% less CO${_2}$ during training and inference with a model up to 64 times smaller than previous approaches

From Personalization to Adaptivity: Creating Immersive Visits through Interactive Digital Storytelling at the Acropolis Museum

Storytelling has recently become a popular way to guide museum visitors, replacing traditional exhibit-centric descriptions by story-centric cohesive narrations with references to the exhibits and multimedia content. This work presents the fundamental elements of the CHESS project approach, the goal of which is to provide adaptive, personalized, interactive storytelling for museum visits. We shortly present the CHESS project and its background, we detail the proposed storytelling and user models, we describe the provided functionality and we outline the main tools and mechanisms employed. Finally, we present the preliminary results of a recent evaluation study that are informing several directions for future work

Personalization in Recommender Systems through Explainable Machine Learning

[Abstract]: Recommender Systems have become ubiquitously utilized tools in multiple fields such as media streaming services, travelling and tourism business, e-commerce, and numerous others. However, in practice they show a tendency to be black-box systems, despite their increasing influence in people’s daily lives. There is a lack of research on providing personalised explanations to the recommendations of a system, that is, integrating the idea of Explainable Artificial Intelligence into the field of Recommender Systems. Therefore, we do not seek to create a Recommender System, but instead devise a way to obtain this explainability or personalisation in such type of tool. In this work, we propose a model able to provide said personalisation by generating explanations based on user-created content, namely text or photographs. In the context of the restaurant review platform TripAdvisor, we will predict, for any (user,restaurant) pair or existing recommendation, the text or image of the restaurant that is most adequate to present said recommendation to the user, that is, the one that best reflects their personal preferences. This model exploits the usage of Matrix Factorisation techniques combined with the feature-rich embeddings of pre-trained image classification and language models (Inception-ResNet-v2 and BERT), to develop a method capable of providing transparency to Recommender Systems.[Resumen]: Los Sistemas de Recomendación se han convertido en herramientas usadas extensivamente en multitud de campos como online streaming, turismo, restauración, viajes y comercio electrónico, así como muchos otros. Sin embargo, en la práctica presentan una tendencia a ser sistemas de caja negra, pese a la cada vez mayor influencia que presentan sobre el día a día de nuestra sociedad. Hay una falta de investigación sobre la idea de aportar explicaciones personalizadas a las recomendaciones de un sistema, es decir, integrar el concepto de Inteligencia Artifical Explicable en el área de los Sistemas de Recomendación. Por lo tanto, no buscamos crear un Sistema de Recomendación per se, sino idear un modo de obtener esta capacidad de explicabilidad o personalización en dicho tipo de sistemas. En este trabajo, proponemos un modelo capaz de proveer de esta personalización mediante la generación de explicaciones basadas en contenido generado por los usuarios, en particular texto e imágenes. En el contexto de la plataforma de reseñas de restaurantes TripAdvisor, buscaremos predecir, para cualquier par o posible recomendación (usuario, restaurante), la imagen o texto sobre dicho restaurante más adecuada para presentar esa recomendación al usuario, es decir, la imagen o texto que mejor refleja las preferencias personales del usuario. Este modelo explota el uso de técnicas de Factorización Matricial combinadas con modelos de lenguaje y clasificación de imágenes (BERT e Inception-ResNet-v2), para desarrollar un método con capacidad de otorgar transparencia a Sistemas de Recomendación.Traballo fin de grao (UDC.FIC). Enxeñaría Informática. Curso 2020/202

Using Semantic-Based User Profile Modeling for Context-Aware Personalised Place Recommendations

Place Recommendation Systems (PRS's) are used to recommend places to visit to World Wide Web users. Existing PRS's are still limited by several problems, some of which are the problem of recommending similar set of places to different users (Lack of Personalization) and no diversity in the set of recommended items (Content Overspecialization). One of the main objectives in the PRS's or Contextual suggestion systems is to fill the semantic gap among the queries and suggestions and going beyond keywords matching. To address these issues, in this study we attempt to build a personalized context-aware place recommender system using semantic-based user profile modeling to address the limitations of current user profile building techniques and to improve the retrieval performance of personalized place recommender system. This approach consists of building a place ontology based on the Open Directory Project (ODP), a hierarchical ontology scheme for organizing websites. We model a semantic user profile from the place concepts extracted from place ontology and weighted according to their semantic relatedness to user interests. The semantic user profile is then exploited to devise a personalized recommendation by re-ranking process of initial search results for improving retrieval performance. We evaluate this approach on dataset obtained using Google Paces API. Results show that our proposed approach significantly improves the retrieval performance compare to classic keyword-based place recommendation model

The Shortest Path to Happiness: Recommending Beautiful, Quiet, and Happy Routes in the City

When providing directions to a place, web and mobile mapping services are all able to suggest the shortest route. The goal of this work is to automatically suggest routes that are not only short but also emotionally pleasant. To quantify the extent to which urban locations are pleasant, we use data from a crowd-sourcing platform that shows two street scenes in London (out of hundreds), and a user votes on which one looks more beautiful, quiet, and happy. We consider votes from more than 3.3K individuals and translate them into quantitative measures of location perceptions. We arrange those locations into a graph upon which we learn pleasant routes. Based on a quantitative validation, we find that, compared to the shortest routes, the recommended ones add just a few extra walking minutes and are indeed perceived to be more beautiful, quiet, and happy. To test the generality of our approach, we consider Flickr metadata of more than 3.7M pictures in London and 1.3M in Boston, compute proxies for the crowdsourced beauty dimension (the one for which we have collected the most votes), and evaluate those proxies with 30 participants in London and 54 in Boston. These participants have not only rated our recommendations but have also carefully motivated their choices, providing insights for future work.Comment: 11 pages, 7 figures, Proceedings of ACM Hypertext 201

Recommender Systems

The ongoing rapid expansion of the Internet greatly increases the necessity of effective recommender systems for filtering the abundant information. Extensive research for recommender systems is conducted by a broad range of communities including social and computer scientists, physicists, and interdisciplinary researchers. Despite substantial theoretical and practical achievements, unification and comparison of different approaches are lacking, which impedes further advances. In this article, we review recent developments in recommender systems and discuss the major challenges. We compare and evaluate available algorithms and examine their roles in the future developments. In addition to algorithms, physical aspects are described to illustrate macroscopic behavior of recommender systems. Potential impacts and future directions are discussed. We emphasize that recommendation has a great scientific depth and combines diverse research fields which makes it of interests for physicists as well as interdisciplinary researchers.Comment: 97 pages, 20 figures (To appear in Physics Reports

Automated illustration of multimedia stories

Submitted in part fulfillment of the requirements for the degree of Master in Computer ScienceWe all had the problem of forgetting about what we just read a few sentences before. This comes from the problem of attention and is more common with children and the elderly. People feel either bored or distracted by something more interesting. The challenge is how can multimedia systems assist users in reading and remembering stories? One solution is to use pictures to illustrate stories as a mean to captivate ones interest as it either tells a story or makes the viewer imagine one. This thesis researches the problem of automated story illustration as a method to increase the readers’ interest and attention. We formulate the hypothesis that an automated multimedia system can help users in reading a story by stimulating their reading memory with adequate visual illustrations. We propose a framework that tells a story and attempts to capture the readers’ attention by providing illustrations that spark the readers’ imagination. The framework automatically creates a multimedia presentation of the news story by (1) rendering news text in a sentence by-sentence fashion, (2) providing mechanisms to select the best illustration for each sentence and (3) select the set of illustrations that guarantees the best sequence. These mechanisms are rooted in image and text retrieval techniques. To further improve users’ attention, users may also activate a text-to-speech functionality according to their preference or reading difficulties. First experiments show how Flickr images can illustrate BBC news articles and provide a better experience to news readers. On top of the illustration methods, a user feedback feature was implemented to perfect the illustrations selection. With this feature users can aid the framework in selecting more accurate results. Finally, empirical evaluations were performed in order to test the user interface,image/sentence association algorithms and users’ feedback functionalities. The respective results are discussed