Design pattern for conversational agents handling data-driven requests

The aim of this research project is to identify design principles for the development of CAs. In the context of this thesis, the research questions are: “According to which design principles are Conversational Agents developed?” and “How can these design principles be meaningfully categorized and described?”. For the aggregation of the design principles, the first step was a systematic literature search according to Vom Brocke et al. (2009). The systematic literature review was followed by a qualitative literature analysis according to Kuckartz (2018). The result of this work is the identification of 15 meta-requirements that could be categorised by means of three main categories and a further seven subcategories. This was followed by the declaration of seven design principles based on the subcategories and their meta-requirements

Secondary Mental Models: Introducing Conversational Agents in Financial Advisory Service Encounters

When introducing unfamiliar Artificial Intelligence (AI)-based systems, such as conversational agents (CAs), one needs to ensure that users interact with them according to their design. While past research has studied single-user environments, many practical settings involve multiple parties. This study addresses this gap and focuses on financial advisory service encounters and how mental models evolve in multi-party contexts. A multimodal interactive CA is developed and tested in financial consultations with 24 clients. The observations of these consultations and subsequent interviews provide insights into the challenges of using CAs in unfamiliar contexts. The clients have difficulties effectively using the system. This is linked to the institutional setting of financial advisory service encounters and a mismatch between the designer’s conceptual model and the client’s mental model, which we call secondary mental model

Secondary Mental Models: Introducing Conversational Agents in Financial Advisory Service Encounters

When introducing unfamiliar Artificial Intelligence (AI)-based systems, such as conversational agents (CAs), one needs to ensure that users interact with them according to their design. While past research has studied single-user environments, many practical settings involve multiple parties. This study addresses this gap and focuses on financial advisory service encounters and how mental models evolve in multi-party contexts. A multimodal interactive CA is developed and tested in financial consultations with 24 clients. The observations of these consultations and subsequent interviews provide insights into the challenges of using CAs in unfamiliar contexts. The clients have difficulties effectively using the system. This is linked to the institutional setting of financial advisory service encounters and a mismatch between the designer’s conceptual model and the client’s mental model, which we call secondary mental model

Explorations in engagement for humans and robots

This paper explores the concept of engagement, the process by which individuals in an interaction start, maintain and end their perceived connection to one another. The paper reports on one aspect of engagement among human interactors--the effect of tracking faces during an interaction. It also describes the architecture of a robot that can participate in conversational, collaborative interactions with engagement gestures. Finally, the paper reports on findings of experiments with human participants who interacted with a robot when it either performed or did not perform engagement gestures. Results of the human-robot studies indicate that people become engaged with robots: they direct their attention to the robot more often in interactions where engagement gestures are present, and they find interactions more appropriate when engagement gestures are present than when they are not.Comment: 31 pages, 5 figures, 3 table

Prompting Datasets: Data Discovery with Conversational Agents

Can large language models assist in data discovery? Data discovery predominantly happens via search on a data portal or the web, followed by assessment of the dataset to ensure it is fit for the intended purpose. The ability of conversational generative AI (CGAI) to support recommendations with reasoning implies it can suggest datasets to users, explain why it has done so, and provide information akin to documentation regarding the dataset in order to support a use decision. We hold 3 workshops with data users and find that, despite limitations around web capabilities, CGAIs are able to suggest relevant datasets and provide many of the required sensemaking activities, as well as support dataset analysis and manipulation. However, CGAIs may also suggest fictional datasets, and perform inaccurate analysis. We identify emerging practices in data discovery and present a model of these to inform future research directions and data prompt design.Comment: 27 pages, 9 figure

Explanation before Adoption: Supporting Informed Consent for Complex Machine Learning and IoT Health Platforms

Explaining health technology platforms to non-technical members of the public is an important part of the process of informed consent. Complex technology platforms that deal with safety-critical areas are particularly challenging, often operating within private domains (e.g. health services within the home) and used by individuals with various understandings of hardware, software, and algorithmic design. Through two studies, the first an interview and the second an observational study, we questioned how experts (e.g. those who designed, built, and installed a technology platform) supported provision of informed consent by participants. We identify a wide range of tools, techniques, and adaptations used by experts to explain the complex SPHERE sensor-based home health platform, provide implications for the design of tools to aid explanations, suggest opportunities for interactive explanations, present the range of information needed, and indicate future research possibilities in communicating technology platforms

A Smart Assistant for Visual Recognition of Painted Scenes

Nowadays, smart devices allow people to easily interact with the surrounding environment thanks to existing communication infrastructures, i.e., 3G/4G/5G or WiFi. In the context of a smart museum, data shared by visitors can be used to provide innovative services aimed to improve their cultural experience. In this paper, we consider as case study the painted wooden ceiling of the Sala Magna of Palazzo Chiaramonte in Palermo, Italy and we present an intelligent system that visitors can use to automatically get a description of the scenes they are interested in by simply pointing their smartphones to them. As compared to traditional applications, this system completely eliminates the need for indoor positioning technologies, which are unfeasible in many scenarios as they can only be employed when museum items are physically distinguishable. Experimental analysis aimed to evaluate the performance of the system in terms of accuracy of the recognition process, and the obtained results show its effectiveness in a real-world application scenario

Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers

Online symptom checkers (OSC) are widely used intelligent systems in health contexts such as primary care, remote healthcare, and epidemic control. OSCs use algorithms such as machine learning to facilitate self-diagnosis and triage based on symptoms input by healthcare consumers. However, intelligent systems’ lack of transparency and comprehensibility could lead to unintended consequences such as misleading users, especially in high-stakes areas such as healthcare. In this paper, we attempt to enhance diagnostic transparency by augmenting OSCs with explanations. We first conducted an interview study (N=25) to specify user needs for explanations from users of existing OSCs. Then, we designed a COVID-19 OSC that was enhanced with three types of explanations. Our lab-controlled user study (N=20) found that explanations can significantly improve user experience in multiple aspects. We discuss how explanations are interwoven into conversation flow and present implications for future OSC designs