Crowdsourcing for Creating a Dataset for Training a Medication Chatbot

To facilitate interaction with mobile health applications, chatbots are increasingly used. They realize the interaction as a dialog where users can ask questions and get answers from the chatbot. A big challenge is to create a comprehensive knowledge base comprising patterns and rules for representing possible user queries the chatbot has to understand and interpret. In this work, we assess how crowdsourcing can be used for generating examples of possible user queries for a medication chatbot. Within one week, the crowdworker generated 2'738 user questions. The examples provide a large variety of possible formulations and information needs. As a next step, these examples for user queries will be used to train our medication chatbot

Improving and Evaluating eMMA's Communication Skills: A Chatbot for Managing Medication

In previous work, a mobile application for medication self-management (eMMA) was introduced. It contained a basic conversational user interface (CUI). In this work, we extended the CUI by integrating the chatbot framework RiveScript and an instruction interface. To study task success, dialog quality and efficiency, we performed a theoretical and a quantitative evaluation as well as a usability test. The results show that the technical extensions of eMMA were useful to improve the chatbot's quality. However, the underlying knowledge base still requires substantial extensions before the system can be used in practice

How to Evaluate Health Applications with Conversational User Interface?

Application of conversational user interfaces (CUI) or chatbots to healthcare is gaining interest fueled by the rising power of artificial intelligence, increasing popularity of mobile health applications and the desire for engagement and usability. While their use is mainly justified by increasing adherence to mobile health applications and facilitating interactions with the system, the question arises: How can such systems be evaluated in a reliable manner? This paper introduces an evaluation framework for health systems whose core interaction principle is a CUI. We derive quality dimensions and attributes by collecting relevant evaluation aspects from applications that have been developed in previous work and from literature on health chatbots. The collected aspects are aggregated into six thematic categories for chatbot quality, including user experience, linguistic, task-oriented and artificial intelligence perspectives, but also healthcare quality and system quality perspectives. The framework is intended to support developers and researchers in the domain of chatbots in healthcare in selecting relevant quality attributes to be assessed before their systems are distributed to patients

Towards Safe Conversational Agents in Healthcare

Conversational agents (CA) are becoming very popular to deliver digital health interventions. These dialog-based systems are interacting with patients using natural language which might lead to misunderstandings and misinterpretations. To avoid patient harm, safety of health CA has to be ensured. This paper raises awareness on safety when developing and distributing health CA. For this purpose, we identify and describe facets of safety and make recommendations for ensuring safety in health CA. We distinguish three facets of safety: 1) system safety, 2) patient safety, and 3) perceived safety. System safety comprises data security and privacy which has to be considered when selecting technologies and developing the health CA. Patient safety is related to risk monitoring and risk management, to adverse events and content accuracy. Perceived safety concerns a user's perception of the level of danger and user's level of comfort during the use. The latter can be supported when data security is guaranteed and relevant information on the system and its capabilities are provided

Can a chatbot increase the motivation to provide personal health information?

In healthcare settings, questionnaires are used to collect information from a patient. A standard method for this are paper-based questionnaires, but they are often complex to understand or long and frustrating to fill. To increase motivation, we developed a chatbot-based system Ana that asks questions that are normally asked using paper forms or in face-to-face encounters. Ana has been developed for the specific use case of collecting the music biography in the context of music therapy. In this paper, we compare user motivation, relevance of answers and time needed to answer the questions depending on the data entry method (i.e. app Ana versus paper-based questionnaire). A randomised trial was performed with 26 students of music therapy. The results show that the chatbot is more motivating and answers are given faster than on paper. No differences in answer relevance could be determined between the two means. We conclude that a chatbot could become an additional data entry method for collecting personal health information

Towards Emotion-Sensitive Conversational User Interfaces in Healthcare Applications

Perception of emotions and adequate responses are key factors of a successful conversational agent. However, determining emotions in a healthcare setting depends on multiple factors such as context and medical condition. Given the increase of interest in conversational agents integrated in mobile health applications, our objective in this work is to introduce a concept for analyzing emotions and sentiments expressed by a person in a mobile health application with a conversational user interface. The approach bases upon bot technology (Synthetic intelligence markup language) and deep learning for emotion analysis. More specifically, expressions referring to sentiments or emotions are classified along seven categories and three stages of strengths using treebank annotation and recursive neural networks. The classification result is used by the chatbot for selecting an appropriate response. In this way, the concerns of a user can be better addressed. We describe three use cases where the approach could be integrated to make the chatbot emotion-sensitive

Microservice chatbot architecture for chronic patient support

Chatbots are able to provide support to patients suffering from very different conditions. Patients with chronic diseases or comorbidities could benefit the most from chatbots which can keep track of their condition, provide specific information, encourage adherence to medication, etc. To perform these functions, chatbots need a suitable underlying software architecture. In this paper, we introduce a chatbot architecture for chronic patient support grounded on three pillars: scalability by means of microservices, standard data sharing models through HL7 FHIR and standard conversation modeling using AIML. We also propose an innovative automation mechanism to convert FHIR resources into AIML files, thus facilitating the interaction and data gathering of medical and personal information that ends up in patient health records. To align the way people interact with each other using messaging platforms with the chatbot architecture, we propose these very same channels for the chatbot-patient interaction, paying special attention to security and privacy issues. Finally, we present a monitored-data study performed in different chronic diseases, and we present a prototype implementation tailored for one specific chronic disease, psoriasis, showing how this new architecture allows the change, the addition or the improvement of different parts of the chatbot in a dynamic and flexible way, providing a substantial improvement in the development of chatbots used as virtual assistants for chronic patients

Validation of a Virtual Assistant for Improving Medication Adherence in Patients with Comorbid Type 2 Diabetes Mellitus and Depressive Disorder

Virtual assistants are programs that interact with users through text or voice messages simulating a human-based conversation. The development of healthcare virtual assistants that use messaging platforms is rapidly increasing. Still, there is a lack of validation of these assistants. In particular, this work aimed to validate the effectiveness of a healthcare virtual assistant, integrated within messaging platforms, with the aim of improving medication adherence in patients with comorbid type 2 diabetes mellitus and depressive disorder. For this purpose, a nine-month pilot study was designed and subsequently conducted. The virtual assistant reminds patients about their medication and provides healthcare professionals with the ability to monitor their patients. We analyzed the medication possession ratio (MPR), measured the level of glycosylated hemoglobin (HbA1c), and obtained the patient health questionnaire (PHQ-9) score in the patients before and after the study. We also conducted interviews with all participants. A total of thirteen patients and five nurses used and evaluated the proposed virtual assistant using the messaging platform Signal. Results showed that on average, the medication adherence improved. In the final interview, 69% of the patients agreed with the idea of continuing to use the virtual assistant after the study

Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare

Evaluating conversational agents (CAs) that are supposed to be applied in healthcare settings and ensuring their quality is essential to avoid patient harm and ensure efficacy of the CA-delivered intervention. However, a guideline for a standardized quality assessment of health CAs is still missing. The objective of this work is to describe a framework that provides guidance for development and evaluation of health CAs. In previous work, consensus on categories for evaluating health CAs has been found. In this work, we identify concrete metrics, heuristics, and checklists for these evaluation categories to form a framework. We focus on a specific type of health CA, namely rule-based systems that are based on written input and output, have a simple personality without any kind of embodiment. First, we identified relevant metrics, heuristics, and checklists to be linked to the evaluation categories through a literature search. Second, five experts judged the metrics regarding their relevance to be considered within evaluation and development of health CAs. The final framework considers nine aspects from a general perspective, five aspects from a response understanding perspective, one aspect from a response generation perspective, and three aspects from an aesthetics perspective. Existing tools and heuristics specifically designed for evaluating CAs were linked to these evaluation aspects (e.g., Bot usability scale, design heuristics for CAs); tools related to mHealth evaluation were adapted when necessary (e.g., aspects from the ISO technical specification for mHealth Apps). The resulting framework comprises aspects to be considered not only as part of a system evaluation, but already during the development. In particular, aspects related to accessibility or security have to be addressed in the design phase (e.g., which input and output options are provided to ensure accessibility?) and have to be verified after the implementation phase. As a next step, transfer of the framework to other types of health CAs has to be studied. The framework has to be validated by applying it during health CA design and development