Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

Background: Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation. Objective: The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care. Methods: We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study. Results: The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes. Conclusions: Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns. Keywords: chatbot; conceptual framework; conversational agent; digital health; mHealth; mobile health; mobile phone

Digital Behavior Change Interventions for the Prevention and Management of Type 2 Diabetes: Systematic Market Analysis

Background Advancements in technology offer new opportunities for the prevention and management of type 2 diabetes. Venture capital companies have been investing in digital diabetes companies that offer digital behavior change interventions (DBCIs). However, little is known about the scientific evidence underpinning such interventions or the degree to which these interventions leverage novel technology-driven automated developments such as conversational agents (CAs) or just-in-time adaptive intervention (JITAI) approaches. Objective Our objectives were to identify the top-funded companies offering DBCIs for type 2 diabetes management and prevention, review the level of scientific evidence underpinning the DBCIs, identify which DBCIs are recognized as evidence-based programs by quality assurance authorities, and examine the degree to which these DBCIs include novel automated approaches such as CAs and JITAI mechanisms. Methods A systematic search was conducted using 2 venture capital databases (Crunchbase Pro and Pitchbook) to identify the top-funded companies offering interventions for type 2 diabetes prevention and management. Scientific publications relating to the identified DBCIs were identified via PubMed, Google Scholar, and the DBCIs’ websites, and data regarding intervention effectiveness were extracted. The Diabetes Prevention Recognition Program (DPRP) of the Center for Disease Control and Prevention in the United States was used to identify the recognition status. The DBCIs’ publications, websites, and mobile apps were reviewed with regard to the intervention characteristics. Results The 16 top-funded companies offering DBCIs for type 2 diabetes received a total funding of US $2.4 billion as of June 15, 2021. Only 4 out of the 50 identified publications associated with these DBCIs were fully powered randomized controlled trials (RCTs). Further, 1 of those 4 RCTs showed a significant difference in glycated hemoglobin A1c (HbA1c) outcomes between the intervention and control groups. However, all the studies reported HbA1c improvements ranging from 0.2% to 1.9% over the course of 12 months. In addition, 6 interventions were fully recognized by the DPRP to deliver evidence-based programs, and 2 interventions had a pending recognition status. Health professionals were included in the majority of DBCIs (13/16, 81%,), whereas only 10% (1/10) of accessible apps involved a CA as part of the intervention delivery. Self-reports represented most of the data sources (74/119, 62%) that could be used to tailor JITAIs. Conclusions Our findings suggest that the level of funding received by companies offering DBCIs for type 2 diabetes prevention and management does not coincide with the level of evidence on the intervention effectiveness. There is considerable variation in the level of evidence underpinning the different DBCIs and an overall need for more rigorous effectiveness trials and transparent reporting by quality assurance authorities. Currently, very few DBCIs use automated approaches such as CAs and JITAIs, limiting the scalability and reach of these solutions

Conversational Agents in Health Care: Scoping Review of Their Behavior Change Techniques and Underpinning Theory

Background Conversational agents (CAs) are increasingly used in health care to deliver behavior change interventions. Their evaluation often includes categorizing the behavior change techniques (BCTs) using a classification system of which the BCT Taxonomy v1 (BCTTv1) is one of the most common. Previous studies have presented descriptive summaries of behavior change interventions delivered by CAs, but no in-depth study reporting the use of BCTs in these interventions has been published to date. Objective This review aims to describe behavior change interventions delivered by CAs and to identify the BCTs and theories guiding their design. Methods We searched PubMed, Embase, Cochrane’s Central Register of Controlled Trials, and the first 10 pages of Google and Google Scholar in April 2021. We included primary, experimental studies evaluating a behavior change intervention delivered by a CA. BCTs coding followed the BCTTv1. Two independent reviewers selected the studies and extracted the data. Descriptive analysis and frequent itemset mining to identify BCT clusters were performed. Results We included 47 studies reporting on mental health (n=19, 40%), chronic disorders (n=14, 30%), and lifestyle change (n=14, 30%) interventions. There were 20/47 embodied CAs (43%) and 27/47 CAs (57%) represented a female character. Most CAs were rule based (34/47, 72%). Experimental interventions included 63 BCTs, (mean 9 BCTs; range 2-21 BCTs), while comparisons included 32 BCTs (mean 2 BCTs; range 2-17 BCTs). Most interventions included BCTs 4.1 “Instruction on how to perform a behavior” (34/47, 72%), 3.3 “Social support” (emotional; 27/47, 57%), and 1.2 “Problem solving” (24/47, 51%). A total of 12/47 studies (26%) were informed by a behavior change theory, mainly the Transtheoretical Model and the Social Cognitive Theory. Studies using the same behavior change theory included different BCTs. Conclusions There is a need for the more explicit use of behavior change theories and improved reporting of BCTs in CA interventions to enhance the analysis of intervention effectiveness and improve the reproducibility of research

Robots to the Rescue: A Review of Studies on Differential Medical Diagnosis Employing Ontology-Based Chat Bot Technology

Access to medical care is a global issue. Technology-aided approaches have been applied in addressing this. Interventions have however not focused on medical diagnosis as a fully automated procedure and available applications employ mainly text-based inputs rather than conversation in natural language. We explored the utility of ontology-based chatbot technology for the design of intelligent agents for medical diagnosis through a systematic review of the most recent related literature. English articles published in 2011-2016 returned 233 hits which yielded 11 relevant articles after a 3-stage screening. Findings showed that the creation of expert systems had been the focus of many the studies which utilize the physician-system-patient framework with system training based mostly on expert knowledge for designing web- or mobile phone-based applications that serve assistive purposes. Findings further indicated gaps in the design and evaluation of more effective systems deployable as standalone applications, for example, on an embodied robotic system. The need for technology supporting the physical examination part of diagnosis, connection to data sources on patients’ vitals and medical history are also indicated in addition to the need for more qualitative work on natural language-based interaction. The system should be one that is continuously learning. Future works should also be directed towards the building of more robust knowledge base as well as evaluation of theory-based diagnostic methodological option

Exploring the Perceptions of mHealth Interventions for the Prevention of Common Mental Disorders in University Students in Singapore: Qualitative Study

Background: Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context. Objective: This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression. Methods: We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework. Results: The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality. Conclusions: Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation

Trends and challenges of e-government chatbots: Advances in exploring open government data and citizen participation content

This work was supported by the Spanish Ministry of Science and Innovation (PID2019-108965GB-I00) and the Regional Government of Andalusia (P20_00314 and B-SEJ-556-UGR20). The authors thank all people who participated in the reported studies.In this paper, we propose a conceptual framework composed of a number of e-government, implementation and evaluation-oriented variables, with which we jointly analyze chatbots presented in the research literature and chatbots deployed as public services in Spain at national, regional and local levels. As a result of our holistic analysis, we identify and discuss current trends and challenges in the development and evaluation of chatbots in the public administration sector, such as focusing the use of the conversational agents on the search for government information, documents and services –leaving citizen consultation and collaboration aside–, and conducting preliminary evaluations of prototypes in limited studies, lacking experiments on deployed systems, with metrics beyond effectiveness and usability –e.g., metrics related to the generation of public values. Addressing some of the identified challenges, we build and evaluate two novel chatbots that present advances in the access to open government data and citizen participation content. Moreover, we come up with additional, potential research lines that may be considered in the future for a new generation of e-government chatbots.Spanish Ministry of Science and Innovation (PID2019-108965GB-I00)Regional Government of Andalusia (P20_00314 and B-SEJ-556-UGR20

Using Chatbots as AI Conversational Partners in Language Learning

Recent advances in Artificial Intelligence (AI) and machine learning have paved the way for the increasing adoption of chatbots in language learning. Research published to date has mostly focused on chatbot accuracy and chatbot–human communication from students’ or in-service teachers’ perspectives. This study aims to examine the knowledge, level of satisfaction and perceptions concerning the integration of conversational AI in language learning among future educators. In this mixed method research based on convenience sampling, 176 undergraduates from two educational settings, Spain (n = 115) and Poland (n = 61), interacted autonomously with three conversational agents (Replika, Kuki, Wysa) over a four-week period. A learning module about Artificial Intelligence and language learning was specifically designed for this research, including an ad hoc model named the Chatbot–Human Interaction Satisfaction Model (CHISM), which was used by teacher candidates to evaluate different linguistic and technological features of the three conversational agents. Quantitative and qualitative data were gathered through a pre-post-survey based on the CHISM and the TAM2 (technology acceptance) models and a template analysis (TA), and analyzed through IBM SPSS 22 and QDA Miner software. The analysis yielded positive results regarding perceptions concerning the integration of conversational agents in language learning, particularly in relation to perceived ease of use (PeU) and attitudes (AT), but the scores for behavioral intention (BI) were more moderate. The findings also unveiled some gender-related differences regarding participants’ satisfaction with chatbot design and topics of interaction.This study is part of a larger research project, [The application of AI and chatbots to language learning], financed by the Instituto de Ciencias de la Educacion at the Univesity of Alicante (Reference number: 5498)

Productivity of incident management with conversational bots-a review

The use of conversational agents (bots) in information systems managed by company’s increases productivity in the development of activities focused on processes such as customer service, healthcare, and presentation. The present work is a systematic literature review that collects articles from 2019 to 2022 in the databases Scopus, Springer, Willey, Indexes-Csic, Taylor & Francis, Pubmed, and Ebsco Host. PRISMA methodology was used to systematize 47 relevant articles. As a result of the analysis, 2/19 very important benefits were obtained, which are: helping to obtain information and facilitating customer service; as for the types of conversational bots, a total of 9 types were found, of which conversational agents and chatbots with artificial intelligence (AI) are the most common; in the case of processes, 3/5 processes that optimize conversational bots were found, where the most prominent are: teaching process, health processes, and customer service processes. An architecture model for conversational bots in incident management is also proposed

Conversational Agents in Education – A Systematic Literature Review

Conversational Agents (CAs) are widely spread in a variety of domains, such as health and customer service. There is a recent trend of increasing publications and implementations of CAs in education. We conduct a systematic literature review to identify common methodologies, pedagogical CA roles, addressed target groups, the technologies and theories behind, as well as human-like design aspects. The initially found 3329 records were systematically reduced to 252 fully coded articles. Based on the analysis of the codings, we derive further research streams. Our results reveal a research gap for long-term studies on the use of CAs in education, and there is insufficient holistic design knowledge for pedagogical CAs. Moreover, target groups other than academic students are rarely considered. We condense our findings in a morphological box and conclude that pedagogical CAs have not yet reached their full potential of long-term practical application in education