Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

Background: Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation. Objective: The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care. Methods: We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study. Results: The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes. Conclusions: Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns. Keywords: chatbot; conceptual framework; conversational agent; digital health; mHealth; mobile health; mobile phone

Exploring the use of conversational agents in healthcare: an overview, conceptual analysis and case study

Conversational agents (CAs) or chatbots are computer programs designed to participate in human-like conversations with their users. The growing interest in these agents is evident in their employment in numerous fields including business, retail, and even healthcare. The need for new initiatives to promote healthy living is apparent in Singapore given its highly stressful and fast-paced way of life, insufficient time for physical activity, and frequent dining out. Singapore is technologically astute and always on the lookout for novel initiatives, such as the use of CAs, to improve patient care and support healthcare staff. This project was grounded in the underlying principles of evidence-based medicine and employed a multitude of study designs and methodologies. Firstly, a (1) scoping review was conducted to map out and evaluate the existing evidence on CAs in healthcare. Secondly, a (2) conceptual framework for rule-based healthcare CA design and development was proposed based on existing mHealth and CA development frameworks. Thirdly, a (3) healthy lifestyle change CA was developed and piloted based on guidance from the proposed conceptual framework and (4) co-design/needs assessment with members of the public through a series of semi-structured interviews. (5) Follow-up interviews were then conducted with participants from the pilot study who reported on their experience interacting with the healthy living CA and their recommendations for its improvement in the future. Finally, a (6) systematic assessment of CA apps was performed in Google Play (android) and Apple (iOS) stores to map out and assess the characteristics and behavior change techniques used in those healthcare CAs currently available to the public for healthy lifestyle behavior change. The results indicated that this field is still in its nascent stages with a predominance of small case studies and a dearth of large-scale randomized controlled trials. Most of the CAs reported in the literature were standalone smartphone apps that were text-based and employed machine learning. The conceptual framework used existing frameworks as a foundation and culminated in a five-step framework guiding the user through defining the aim, creating an identity, developing the content, building the conversation flow, and piloting the CA. The CA designed and developed for a healthy lifestyle change, Precilla, aimed to include those recommendations made by volunteer participants in the needs assessment interviews. Their suggestions included providing educational information on diabetes and prediabetes, risk factors, and risk prevention. They also suggested a friendly personality and simple, layman terms to be used in the conversations with Precilla. Some participants were very satisfied with Precilla, found her informative, and thought the information shared was very valuable and a good reminder to make healthier choices with regards to healthy eating, physical activity, sleep, and stress. Conversely, others were disappointed in the lack of artificial intelligence and would have liked to have the opportunity to “chit-chat” with Precilla without any specific endpoint. One of the main recommendations for improvement suggested introducing some personalization to the CA in terms of more tailored content and message delivery timings. Incentives such as rewards and point collection were also of interest and participants had high regard for formal and informal group support in enabling them to achieve their healthy lifestyle change goals. Finally, the systematic assessment of healthy living apps from the app stores indicated that the pool of CA apps in this area is fairly small. The apps focus on just one area of healthy living either diabetes prevention, wellness and physical activity, sleep improvements, or healthier eating. The apps were all free with only a couple offering in-app purchases and no advertisements were featured. CA personality was sometimes neglected in most apps which came across as very rigid, factual, and not human-like. This Ph.D. project has made several valuable contributions such as a timely update of the current progress and status of this field of research on healthcare CAs through the review and synthesis of evidence mapped in the scoping review and apps assessment. It has also proposed some guidelines on CA development which currently have not been strictly defined. Finally, a test of the feasibility and acceptability of a healthcare conversational agent for healthy living in Singapore revealed the applicability and/or necessity of such an agent and could propose future potential applications in Singapore’s healthcare system. This project has laid a good foundation for future research to tap into.Doctor of Philosoph

Public perceptions of a healthy lifestyle change conversational agent in Singapore: a qualitative study

Objective: Conversational agents (CAs) are increasingly used for the delivery of healthy lifestyle behaviour interventions. This qualitative study aimed to explore the barriers and facilitators to participants’ usage of a healthy lifestyle change CA and collect their views on areas for its improvement. Methods: Twenty participants were recruited from a convenience sample of users interacting with a CA promoting healthy lifestyle changes to the general population in Singapore. This CA, Precilla, educated users on healthy living, specifically: diet, exercise, sleep and stress; for four weeks. The volunteers participated in semi-structured interviews where an interview guide was used, with questions on acceptability, satisfaction and critical appraisal of the CA. Interviews were transcribed and analysed in parallel by two researchers using thematic content analysis. Results: Four main themes were identified: (1) enjoyable and acceptable experiences, (2) suboptimal experience(s), (3) alterations to Precilla for enhanced interaction and (4) suggestions for the future. Enjoyable experiences referenced the CA's friendly personality and important content that motivated a positive change to their lifestyle. Some participants were less satisfied and found the content to be too simple or sometimes, the messages too lengthy. Conclusions: Participants suggested that in the future, CAs should provide regularly updated content on healthy living, specifically pre-diabetes. Multiple answer options should also be provided for more personalisation along with links to external resources to help improve users’ health literacy. Further recommendations include a necessity for a user-centered approach in CA development, employment of engagement strategies, use of a delivery platform most familiar to the target population and stratified message timings to suit the population and purpose of CA. Translating the health CAs to languages relevant to the target group could also enable wider reach and applicability.Published versio

Conversational Agent for Healthy Lifestyle Behavior Change: Web-Based Feasibility Study

BackgroundThe rising incidence of chronic diseases is a growing concern, especially in Singapore, which is one of the high-income countries with the highest prevalence of diabetes. Interventions that promote healthy lifestyle behavior changes have been proven to be effective in reducing the progression of prediabetes to diabetes, but their in-person delivery may not be feasible on a large scale. Novel technologies such as conversational agents are a potential alternative for delivering behavioral interventions that promote healthy lifestyle behavior changes to the public. ObjectiveThe aim of this study is to assess the feasibility and acceptability of using a conversational agent promoting healthy lifestyle behavior changes in the general population in Singapore. MethodsWe performed a web-based, single-arm feasibility study. The participants were recruited through Facebook over 4 weeks. The Facebook Messenger conversational agent was used to deliver the intervention. The conversations focused on diet, exercise, sleep, and stress and aimed to promote healthy lifestyle behavior changes and improve the participants’ knowledge of diabetes. Messages were sent to the participants four times a week (once for each of the 4 topics of focus) for 4 weeks. We assessed the feasibility of recruitment, defined as at least 75% (150/200) of our target sample of 200 participants in 4 weeks, as well as retention, defined as 33% (66/200) of the recruited sample completing the study. We also assessed the participants’ satisfaction with, and usability of, the conversational agent. In addition, we performed baseline and follow-up assessments of quality of life, diabetes knowledge and risk perception, diet, exercise, sleep, and stress. ResultsWe recruited 37.5% (75/200) of the target sample size in 1 month. Of the 75 eligible participants, 60 (80%) provided digital informed consent and completed baseline assessments. Of these 60 participants, 56 (93%) followed the study through till completion. Retention was high at 93% (56/60), along with engagement, denoted by 50% (30/60) of the participants communicating with the conversational agent at each interaction. Acceptability, usability, and satisfaction were generally high. Preliminary efficacy of the intervention showed no definitive improvements in health-related behavior. ConclusionsThe delivery of a conversational agent for healthy lifestyle behavior change through Facebook Messenger was feasible and acceptable. We were unable to recruit our planned sample solely using the free options in Facebook. However, participant retention and conversational agent engagement rates were high. Our findings provide important insights to inform the design of a future randomized controlled trial

Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

Background: Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation. Objective: The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care. Methods: We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study. Results: The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations—user-centered design and privacy and security—that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes. Conclusions: Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns.ISSN:2291-522