Visualizing recommendations to support exploration, transparency and controllability

Research on recommender systems has traditionally focused on the development of algorithms to improve accuracy of recommendations. So far, little research has been done to enable user interaction with such systems as a basis to support exploration and control by end users. In this paper, we present our research on the use of information visualization techniques to interact with recommender systems. We investigated how information visualization can improve user understanding of the typically black-box rationale behind recommendations in order to increase their perceived relevance and meaning and to support exploration and user involvement in the recommendation process. Our study has been performed using TalkExplorer, an interactive visualization tool developed for attendees of academic conferences. The results of user studies performed at two conferences allowed us to obtain interesting insights to enhance user interfaces that integrate recommendation technology. More specifically, effectiveness and probability of item selection both increase when users are able to explore and interrelate multiple entities - i.e. items bookmarked by users, recommendations and tags. Copyright © 2013 ACM

Layered evaluation of multi-criteria collaborative filtering for scientific paper recommendation

Recommendation algorithms have been researched extensively to help people deal with abundance of information. In recent years, the incorporation of multiple relevance criteria has attracted increased interest. Such multi-criteria recommendation approaches are researched as a paradigm for building intelligent systems that can be tailored to multiple interest indicators of end-users – such as combinations of implicit and explicit interest indicators in the form of ratings or ratings on multiple relevance dimensions. Nevertheless, evaluation of these recommendation techniques in the context of real-life applications still remains rather limited. Previous studies dealing with the evaluation of recommender systems have outlined that the performance of such algorithms is often dependent on the dataset – and indicate the importance of carrying out careful testing and parameterization. Especially when looking at large scale datasets, it becomes very difficult to deploy evaluation methods that may help in assessing the effect that different system components have to the overall design. In this paper, we study how layered evaluation can be applied for the case of a multi-criteria recommendation service that we plan to deploy for paper recommendation using the Mendeley dataset. The paper introduces layered evaluation and suggests two experiments that may help assess the components of the envisaged system separately. Keywords: Recommender systems; Multi-Criteria Decision Making (MCDM); Evaluatio

IntersectionExpIorer: The flexibility of multiple perspectives

Recommender systems are currently an ubiquitous presence on the web, helping us find relevant items in the ever-growing plethora of information available. However, there is not a one-size fits-all for recommender systems, and flexibility and control are crucial for enabling the possibility of adapting the recommender system to different user preferences. In this paper, we present the results of a study designed to assess user interaction with IntersectionExplorer (IEx), a multi-perspective tool for exploring conference paper recommendations. The study was conducted at the Digital Humanities 2016 Conference, an event with a rather large, heterogeneous, and not technology-oriented audience. The results obtained indicate that the IEx multi-perspective approach lends enough flexibility to accommodate different user preferences. When contrasting these results with a previous study conducted at a conference with a highly technological audience, it becomes apparent that the flexibility of IEx is key to empower users with different profiles to customize their approach to finding relevant recommendations

The effect of different set-based visualizations on user exploration of recommendations

When recommendations fail, trust in a recommender system often decreases, particularly when the system acts like a "black box". To deal with this issue, it is important to support exploration of recommendations by explicitly exposing relationships that can provide explanations. As an example, a graph-based visualization can help to explain collaborative filtering results by representing relationships among items and users. In our work, we focus on the use of visualization techniques to support exploration of multiple relevance prospects - such as relationships between different recommendation methods, socially connected users and tags. More specifically, we researched how users explore relationships between such multiple relevance prospects with two set-based visualization techniques: a clustermap and a Venn diagram. A comparative analysis of user studies with these two approaches indicates that, although effectiveness of recommendations increases with the use of a clustermap, the approach is too complex for a non-technical audience. A Venn diagram representation is more intuitive and users are more likely to explore relationships that help them find relevant items

<i>“We’re Seeking Relevance”</i>: Qualitative Perspectives on the Impact of Learning Analytics on Teaching and Learning

Whilst a significant body of learning analytics research tends to focus on impact from the perspective of usability or improved learning outcomes, this paper proposes an approach based on Affordance Theory to describe awareness and intention as a bridge between usability and impact. 10 educators at 3 European institutions participated in detailed interviews on the affordances they perceive in using learning analytics to support practice in education. Evidence illuminates connections between an educator’s epistemic beliefs about learning and the purpose of education, their perception of threats or resources in delivering a successful learning experience, and the types of data they would consider as evidence in recognising or regulating learning. This evidence can support the learning analytics community in considering the proximity to the student, the role of the educator, and their personal belief structure in developing robust analytics tools that educators may be more likely to use

Preface to Proceedings of the 1st Workshop on Recommender Systems in Technology Enhanced Learning (RecSysTEL 2010)

AbstractTechnology enhanced learning (TEL) aims to design, develop and test socio-technical innovations that will support and enhance learning practices of both individuals and organisations. It is an application domain that generally addresses all types of technology research & development aiming to support teaching and learning activities. Information retrieval is a pivotal activity in TEL, and the deployment of recommender systems has attracted increased interest during the past years.Recommendation methods, techniques and systems open an interesting new approach to facilitate and support learning and teaching. There are plenty of resources available on the Web, both in terms of digital learning content and people resources (e.g. other learners, experts, tutors) that can be used to facilitate teaching and learning tasks. The challenge is to develop, deploy and evaluate systems that provide learners and teachers with meaningful guidance in order to help identify suitable learning resources from a potentially overwhelming variety of choices.The 1st Workshop on Recommender Systems for Technology Enhanced Learning (RecSysTEL) builds upon the tradition of a series of workshops on Social Information Retrieval for Technology Enhanced Learning (SIRTEL), Context-Aware Recommendation for Learning and Towards User Modelling and Adaptive Systems for All (TUMAS-A)a. RecSysTEL was organised jointly by the 4th ACM Conference on Recommender Systems (RecSys 2010) and the 5th European Conference on Technology Enhanced Learning (EC-TEL 2010), on 29–30 September 2010 in Barcelona, Spain. Its main goal was to bring together researchers and practitioners who are working on topics related to the design, development and testing of recommender systems in educational settings as well as present the current status of research in this area and create cross-disciplinary liaisons between the RecSys and ECTEL communities. Overall, its contributions outline the rich potential of TEL as an application area for recommender systems and identify the challenges of developing such systems in a TEL context

Supporting conference attendees with visual decision making interfaces

Recent efforts in recommender systems research focus increasingly on human factors affecting recommendation acceptance, such as transparency and user control. In this paper, we present IntersectionExplorer, a scalable visualization to interleave the output of several recommender engines with user-contributed relevance information, such as bookmarks and tags. Two user studies at conferences indicate that this approach is well suited for technical audiences in smaller venues, and allowed the identification of applicability limitations for less technical audiences attending larger events. Copyright held by the owner/author(s)

Panorama of Recommender Systems to Support Learning

This chapter presents an analysis of recommender systems in TechnologyEnhanced Learning along their 15 years existence (2000-2014). All recommender systems considered for the review aim to support educational stakeholders by personalising the learning process. In this meta-review 82 recommender systems from 35 different countries have been investigated and categorised according to a given classification framework. The reviewed systems have been classified into 7 clusters according to their characteristics and analysed for their contribution to the evolution of the RecSysTEL research field. Current challenges have been identified to lead the work of the forthcoming years.Hendrik Drachsler has been partly supported by the FP7 EU Project LACE (619424). Katrien Verbert is a post-doctoral fellow of the Research Foundation Flanders (FWO). Olga C. Santos would like to acknowledge that her contributions to this work have been carried out within the project Multimodal approaches for Affective Modelling in Inclusive Personalized Educational scenarios in intelligent Contexts (MAMIPEC -TIN2011-29221-C03-01). Nikos Manouselis has been partially supported with funding CIP-PSP Open Discovery Space (297229

Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI

While research on explainable AI (XAI) is booming and explanation techniques have proven promising in many application domains, standardised human-centred evaluation procedures are still missing. In addition, current evaluation procedures do not assess XAI methods holistically in the sense that they do not treat explanations' effects on humans as a complex user experience. To tackle this challenge, we propose to adapt the User-Centric Evaluation Framework used in recommender systems: we integrate explanation aspects, summarise explanation properties, indicate relations between them, and categorise metrics that measure these properties. With this comprehensive evaluation framework, we hope to contribute to the human-centred standardisation of XAI evaluation.Comment: This preprint has not undergone any post-submission improvements or corrections. This work was an accepted contribution at the XAI world Conference 202