Evaluating Conversational Recommender Systems: A Landscape of Research

Conversational recommender systems aim to interactively support online users in their information search and decision-making processes in an intuitive way. With the latest advances in voice-controlled devices, natural language processing, and AI in general, such systems received increased attention in recent years. Technically, conversational recommenders are usually complex multi-component applications and often consist of multiple machine learning models and a natural language user interface. Evaluating such a complex system in a holistic way can therefore be challenging, as it requires (i) the assessment of the quality of the different learning components, and (ii) the quality perception of the system as a whole by users. Thus, a mixed methods approach is often required, which may combine objective (computational) and subjective (perception-oriented) evaluation techniques. In this paper, we review common evaluation approaches for conversational recommender systems, identify possible limitations, and outline future directions towards more holistic evaluation practices

Explanations in Music Recommender Systems in a Mobile Setting

Revised version: some spelling errors corrected.Every day, millions of users utilize their mobile phones to access music streaming services such as Spotify. However, these `black boxes’ seldom provide adequate explanations for their music recommendations. A systematic literature review revealed that there is a strong relationship between moods and music, and that explanations and interface design choices can effect how people perceive recommendations just as much as algorithm accuracy. However, little seems to be known about how to apply user-centric design approaches, which exploit affective information to present explanations, to mobile devices. In order to bridge these gaps, the work of Andjelkovic, Parra, & O’Donovan (2019) was extended upon and applied as non-interactive designs in a mobile setting. Three separate Amazon Mechanical Turk studies asked participants to compare the same three interface designs: baseline, textual, and visual (n=178). Each survey displayed a different playlist with either low, medium, or high music popularity. Results indicate that music familiarity may or may not influence the need for explanations, but explanations are important to users. Both explanatory designs fared equally better than the baseline, and the use of affective information may help systems become more efficient, transparent, trustworthy, and satisfactory. Overall, there does not seem to be a `one design fits all’ solution for explanations in a mobile setting.Master's Thesis in Information ScienceINFO390MASV-INFOMASV-IK

Considering the User in the Wireless World

The near future promises significant advances in communication capabilities, but one of the keys to success is the capability understanding of the people with regards to its value and usage. In considering the role of the user in the wireless world of the future, the Human Perspective Working Group (WG1) of the Wireless World Research Forum has gathered input and developed positions in four important areas: methods, processes, and best practices for user-centered research and design; reference frameworks for modeling user needs within the context of wireless systems; user scenario creation and analysis; and user interaction technologies. This article provides an overview of WG1's work in these areas that are critical to ensuring that the future wireless world meets and exceeds the expectations of people in the coming decades

QueryTogether: Enabling entity-centric exploration in multi-device collaborative search

Collaborative and co-located information access is becoming increasingly common. However, fairly little attention has been devoted to the design of ubiquitous computing approaches for spontaneous exploration of large information spaces enabling co-located collaboration. We investigate whether an entity-based user interface provides a solution to support co-located search on heterogeneous devices. We present the design and implementation of QueryTogether, a multi-device collaborative search tool through which entities such as people, documents, and keywords can be used to compose queries that can be shared to a public screen or specific users with easy touch enabled interaction. We conducted mixed-methods user experiments with twenty seven participants (nine groups of three people), to compare the collaborative search with QueryTogether to a baseline adopting established search and collaboration interfaces. Results show that QueryTogether led to more balanced contribution and search engagement. While the overall s-recall in search was similar, in the QueryTogether condition participants found most of the relevant results earlier in the tasks, and for more than half of the queries avoided text entry by manipulating recommended entities. The video analysis demonstrated a more consistent common ground through increased attention to the common screen, and more transitions between collaboration styles. Therefore, this provided a better fit for the spontaneity of ubiquitous scenarios. QueryTogether and the corresponding study demonstrate the importance of entity based interfaces to improve collaboration by facilitating balanced participation, flexibility of collaboration styles and social processing of search entities across conversation and devices. The findings promote a vision of collaborative search support in spontaneous and ubiquitous multi-device settings, and better linking of conversation objects to searchable entities

User interface patterns in recommendation-empowered content intensive multimedia applications

Design Patterns (DPs) are acknowledged as powerful conceptual tools to improve design quality and to reduce time and cost of the development process by effect of the reuse of “good” design solutions. In many fields (e.g., software engineering, web engineering, interface design) patterns are widely used by practitioners and are also investigated from a research perspective. Still, they have been seldom explored in the arena of Recommender Systems (RSs). RSs provide suggestions (“recommendations”) for items that are likely to be appropriate for the user profile, and are increasingly adopted in content-intensive multimedia applications to complement traditional forms of search in large information spaces. This paper explores RSs through the lens of User Interface (UI) Design Patterns. We have performed a systematic analysis of 54 recommendation-empowered content-intensive multimedia applications, in order to: (i) discover the occurrences of existing domain independent UI patterns; (ii) identify frequently adopted UI solutions that are not modelled by existing patterns, and define a set of new UI patterns, some of which are specific of the interfaces for recommendation features while others can be useful also in a broader context. The results of our inspection have been discussed with and evaluated by a team of experts, leading to a consolidated set of 14 new patterns that are reported in the paper. Reusing pattern-based design solutions instead of building new solutions from scratch enables novice and expert designers to build good UIs for Recommendation-empowered content intensive multimedia applications more effectively, and ultimately can improve the UX experience in this class of systems. From a broader perspective, our work can stimulate future research bridging Recommender Systems, Web Engineering and Interface Design by means of Design Patterns, and highlights new research directions also discussed in the paper