Health professions digital education on clinical practice guidelines: a systematic review by Digital Health Education collaboration

Background Clinical practice guidelines are an important source of information, designed to help clinicians integrate research evidence into their clinical practice. Digital education is increasingly used for clinical practice guideline dissemination and adoption. Our aim was to evaluate the effectiveness of digital education in improving the adoption of clinical practice guidelines. Methods We performed a systematic review and searched seven electronic databases from January 1990 to September 2018. Two reviewers independently screened studies, extracted data and assessed risk of bias. We included studies in any language evaluating the effectiveness of digital education on clinical practice guidelines compared to other forms of education or no intervention in healthcare professionals. We used the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach to assess the quality of the body of evidence. Results Seventeen trials involving 2382 participants were included. The included studies were diverse with a largely unclear or high risk of bias. They mostly focused on physicians, evaluated computer-based interventions with limited interactivity and measured participants’ knowledge and behaviour. With regard to knowledge, studies comparing the effect of digital education with no intervention showed a moderate, statistically significant difference in favour of digital education intervention (SMD = 0.85, 95% CI 0.16, 1.54; I2 = 83%, n = 3, moderate quality of evidence). Studies comparing the effect of digital education with traditional learning on knowledge showed a small, statistically non-significant difference in favour of digital education (SMD = 0.23, 95% CI − 0.12, 0.59; I2 = 34%, n = 3, moderate quality of evidence). Three studies measured participants’ skills and reported mixed results. Of four studies measuring satisfaction, three studies favoured digital education over traditional learning. Of nine studies evaluating healthcare professionals’ behaviour change, only one study comparing email-delivered, spaced education intervention to no intervention reported improvement in the intervention group. Of three studies reporting patient outcomes, only one study comparing email-delivered, spaced education games to non-interactive online resources reported modest improvement in the intervention group. The quality of evidence for outcomes other than knowledge was mostly judged as low due to risk of bias, imprecision and/or inconsistency. Conclusions Health professions digital education on clinical practice guidelines is at least as effective as traditional learning and more effective than no intervention in terms of knowledge. Most studies report little or no difference in healthcare professionals’ behaviours and patient outcomes. The only intervention shown to improve healthcare professionals’ behaviour and modestly patient outcomes was email-delivered, spaced education. Future research should evaluate interactive, simulation-based and spaced forms of digital education and report on outcomes such as skills, behaviour, patient outcomes and cost

Smartphone apps for point-of-care information summaries: systematic assessment of the quality and content

Background Clinicians need easy access to evidence-based information to inform their clinical practice. Point-of-care information summaries are increasingly available in the form of smartphone apps. However, the quality of information from the apps is questionable as there is currently no regulation on the content of the medical apps. Objectives This study aimed to systematically assess the quality and content of the medical apps providing point-of-care information summaries that were available in two major app stores. We evaluated apps designed specifically for healthcare professionals and assessed their content development, editorial policy, coverage of medical conditions and trustworthiness. Methods We conducted a systematic assessment of medical apps providing point-of-care information summaries available in Google Play and Apple app stores. Apps launched or updated since January 2020 were identified through a systematic search using 42matters. Apps meeting the inclusion criteria were downloaded and assessed. The data extraction and app assessment were done in parallel and independently by at least two reviewers. Apps were evaluated against the adapted criteria: (1) general characteristics, (2) content presentation of the summaries, (3) editorial quality, (4) evidence-based methodology, (5) coverage (volume) of the medical conditions, (6) usability of apps and (7) trustworthiness of the app based on HONcode principles. HONcode principles are guidelines used to inform users about the credibility and reliability of health information online. The results were reported as a narrative review. Results Eight medical apps met the inclusion criteria and were systematically appraised. Based on our evaluation criteria, UpToDate supported 16 languages, and all other apps were English. Bullet points and brief paragraphs were used in all apps, and only DynaMed and Micromedex and Pathway-medical knowledge provided a formal grading system for the strength of recommendations for all the medical conditions in their apps. All the other apps either lacked a formal grading system altogether or offered one for some of the medical conditions. About 30% of the editorial quality assessment and 47.5% of the evidence-based methodology assessment were unclear or missing. UpToDate contained the most point-of-care evidence-based documents with >10 500 documents. All apps except 5-Minute Clinical Consult and DynaMed and Micromedex were available for offline access. Only Medscape complied with the HONcode principles. Conclusions Future apps should report a more detailed evidence-based methodology, be accessible for offline use and support search in more than one language. There should be clearer information provided in future apps regarding the declaration of authorship and conflict of interest

Conversational Agents in Health Care: Expert Interviews to Inform the Definition, Classification, and Conceptual Framework

Background Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in health care settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes and unreliable research findings. We developed and published a conceptual framework, designing, developing, evaluating, and implementing a smartphone-delivered, rule-based conversational agent (DISCOVER), consisting of 3 iterative stages of CA design, development, and evaluation and implementation, complemented by 2 cross-cutting themes (user-centered design and data privacy and security). Objective This study aims to perform in-depth, semistructured interviews with multidisciplinary experts in health care CAs to share their views on the definition and classification of health care CAs and evaluate and validate the DISCOVER conceptual framework. Methods We conducted one-on-one semistructured interviews via Zoom (Zoom Video Communications) with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed using thematic analysis. Results Following participants’ input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using ≥1 communication modality, such as text, voice, images, or video. CAs were classified by 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of health care intervention. Experts considered that the conceptual framework could be adapted for artificial intelligence–based CAs. However, despite recent advances in artificial intelligence, including large language models, the technology is not able to ensure safety and reliability in health care settings. Finally, aligned with participants’ feedback, we present an updated iteration of the conceptual framework for health care conversational agents (CHAT) with key considerations for CA design, development, and evaluation and implementation, complemented by 3 cross-cutting themes: ethics, user involvement, and data privacy and security. Conclusions We present an expanded, validated CHAT and aim at guiding researchers from a variety of backgrounds and with different levels of expertise in the design, development, and evaluation and implementation of rule-based CAs in health care settings

Offline eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction

Background: The world is short of 7.2 million health–care workers and this figure is growing. The shortage of teachers is even greater, which limits traditional education modes. eLearning may help overcome this training need. Offline eLearning is useful in remote and resource–limited settings with poor internet access. To inform investments in offline eLearning, we need to establish its effectiveness in terms of gaining knowledge and skills, students’ satisfaction and attitudes towards eLearning. Methods: We conducted a systematic review of offline eLearning for students enrolled in undergraduate, health–related university degrees. We included randomised controlled trials that compared offline eLearning to traditional learning or an alternative eLearning method. We searched the major bibliographic databases in August 2013 to identify articles that focused primarily on students’ knowledge, skills, satisfaction and attitudes toward eLearning, and health economic information and adverse effects as secondary outcomes. We also searched reference lists of relevant studies. Two reviewers independently extracted data from the included studies. We synthesized the findings using a thematic summary approach. Findings: Forty–nine studies, including 4955 students enrolled in undergraduate medical, dentistry, nursing, psychology, or physical therapy studies, met the inclusion criteria. Eleven of the 33 studies testing knowledge gains found significantly higher gains in the eLearning intervention groups compared to traditional learning, whereas 21 did not detect significant differences or found mixed results. One study did not test for differences. Eight studies detected significantly higher skill gains in the eLearning intervention groups, whilst the other 5 testing skill gains did not detect differences between groups. No study found offline eLearning as inferior. Generally no differences in attitudes or preference of eLearning over traditional learning were observed. No clear trends were found in the comparison of different modes of eLearning. Most of the studies were small and subject to several biases. Conclusions: Our results suggest that offline eLearning is equivalent and possibly superior to traditional learning regarding knowledge, skills, attitudes and satisfaction. Although a robust conclusion cannot be drawn due to variable quality of the evidence, these results justify further investment into offline eLearning to address the global health care workforce shortage

Exploring views and experiences of the general public’s adoption of digital technologies for healthy lifestyle in Singapore: a qualitative study

ObjectiveLittle is known about the general adult population’s adoption of digital technology to support healthy lifestyle, especially when they are expected to take greater personal responsibility for managing their health and well-being today. The current qualitative study intended to gain an in-depth understanding of determinants of digital technology adoption for healthy lifestyle among community-dwelling adults in Singapore.DesignA qualitative study design, with thematic framework analysis was applied to develop themes from the data.SettingSemi-structured individual interviews were conducted with participants either face-to-face or online through a videoconferencing platform.Participants14 women and 16 men from the general population who were between the ages of 22 and 71 years.ResultsThree major themes were developed: (1) digitally disempowered (2) safety and perceived risks and harm; (3) cultural values and drives. Adoption of technology among the general population is needs-driven, and contingent on individual, technological and other cross-cultural contextual factors.ConclusionOur findings highlight there is no one solution which fits all individuals, emphasizing the challenges of catering to diverse groups to reduce barriers to adoption of digital technologies for healthy lifestyle. Digital guidance and training, as well as social influences, can motivate technological adoption in the population. However, technical problems as well as data security and privacy concerns should first be adequately addressed. This study provides rich cross-cultural insights and informs policy-making due to its alignment with government public health initiatives to promote healthy lifestyle

Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

Background: Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation. Objective: The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care. Methods: We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study. Results: The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes. Conclusions: Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns. Keywords: chatbot; conceptual framework; conversational agent; digital health; mHealth; mobile health; mobile phone

LOWER URINARY TRACT SYMPTOMS AND DEPRESSION IN PATIENTS WITH MULTIPLE SCLEROSIS

Background: Both depression and lower urinary tract symptoms (LUTS) may be present in patients with multiple sclerosis (MS). The objective of this study was to give an insight on depression and LUTS in patients with MS in Croatia and to determine the possible association between LUTS and depression in patients with MS. Subjects and methods: This was a prospective cross-sectional study conducted in a tertiary healthcare center in Croatia. Hundred and one consecutive patients with MS (75 female, 26 male, mean age 42.09 (range 19-77) years, mean Expanded Disability Status Scale (EDSS) score 3.1 (range 0.0-7.0)) participated in this study. We evaluated LUTS and related quality of life (QoL) using three International Consultation on Incontinence Questionnaires (ICIQ) enquiring about overactive bladder (ICIQ-OAB), urinary incontinence short form (ICIQ-UI SF) and lower urinary tract symptoms related quality of life (ICIQLUTS-QoL). ICIQ-OAB and ICIQLUTS-QoL were for this purpose with permission successfully translated and validated into Croatian, while ICIQ-UI SF was already previously validated for the Croatian language. Information regarding treatment for depression was obtained during the medical interview. Data were analyzed and interpreted using IBM SPSS Statistics for Windows, version 23.0 (IBM Corp., Armonk, N.Y., USA). Results: 89.10% (N=90) patients with MS reported urgency with urge urinary incontinence (UUI) present in 70.29% (N=71). 81.18% (N=82) patients reported nocturia, and 90.09% (N=91) reported feeling drowsy or sleepy during the day due to bladder symptoms. Neurological deficit measured by EDSS was found to positively correlate with LUTS on all three questionnaires: ICIQOAB (r=0.390, p<0.05), ICIQ-UI SF (r=0.477, p<0.01) and ICIQ-LUTSQoL (r=0.317, p<0.05). 25 patients were in treatment for depression. There were no significant differences between female and male patients regarding treatment for depression (2=0.018, df=1, p>0.05). Results on ICIQ-UI SF showed that depressive patients had more pronounced LUTS (t=2.067, df=99, p<0.05), which was also true for the ICIQ-LUTSQoL (t=-2.193, df=99, p<0.05). Positive correlations were found between depression and LUTS on ICIQ-UI SF (r=0.203, p<0.05) and ICIQ-LUTSQoL (r=0.215, p<0.05). Conclusion: This study gives insight into the presence of depression and LUTS in Croatian patients with MS for which purpose ICIQ-OAB and ICIQ-LUTSQoL were with permission successfully translated and validated into Croatian. The connection between depression and LUTS must be considered when managing patients with MS

Digital Education for Health Professions on Smoking Cessation Management:Systematic Review by the Digital Health Education Collaboration

BACKGROUND: Tobacco smoking, one of the leading causes of preventable death and disease, is associated with 7 million deaths every year. This is estimated to rise to more than 8 million deaths per year by 2030, with 80% occurring in low- and middle-income countries. Digital education, teaching, and learning using digital technologies have the potential to increase educational opportunities, supplement teaching activities, and decrease distance barriers in health professions education. OBJECTIVE: The primary objective of this systematic review was to evaluate the effectiveness of digital education compared with various controls in improving learners' knowledge, skills, attitudes, and satisfaction to deliver smoking cessation therapy. The secondary objectives were to assess patient-related outcomes, change in health professionals' practice or behavior, self-efficacy or self-rated competence of health professionals in delivering smoking cessation therapy, and cost-effectiveness of the interventions. METHODS: We searched 7 electronic databases and 2 trial registers for randomized controlled trials published between January 1990 and August 2017. We used gold standard Cochrane methods to select and extract data and appraise eligible studies. RESULTS: A total of 11 studies (number of participants, n=2684) were included in the review. All studies found that digital education was at least as effective as traditional or usual learning. There was some suggestion that blended education results in similar or greater improvements in knowledge (standardized mean difference, SMD=0.19, 95% CI -0.35 to 0.72), skill (SMD=0.58, 95% CI 0.08-1.08), and satisfaction (SMD=0.62, 95% CI 0.12-1.12) compared with digital education or usual learning alone. There was also some evidence for improved attitude (SMD=0.45, 95% CI 0.18-0.72) following digital education compared with usual learning. Only 1 study reported patient outcomes and the setup cost of blended education but did not compare outcomes among groups. There were insufficient data to investigate what components of the digital education interventions were associated with the greatest improvements in learning outcomes. CONCLUSIONS: The evidence suggests that digital education is at least as effective as usual learning in improving health professionals' knowledge and skill for delivering smoking cessation therapy. However, limitations in the evidence base mean that these conclusions should be interpreted with some caution. TRIAL REGISTRATION: PROSPERO CRD42016046815; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=46815

Exploring the Perceptions of mHealth Interventions for the Prevention of Common Mental Disorders in University Students in Singapore: Qualitative Study

Background: Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context. Objective: This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression. Methods: We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework. Results: The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality. Conclusions: Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation

Serious gaming and gamification education in health professions: Systematic review

Background: There is a worldwide shortage of health workers, and this issue requires innovative education solutions. Serious gaming and gamification education have the potential to provide a quality, cost-effective, novel approach that is flexible, portable, and enjoyable and allow interaction with tutors and peers. Objective: The aim of this systematic review was to evaluate the effectiveness of serious gaming/gamification for health professions education compared with traditional learning, other types of digital education, or other serious gaming/gamification interventions in terms of patient outcomes, knowledge, skills, professional attitudes, and satisfaction (primary outcomes) as well as economic outcomes of education and adverse events (secondary outcomes). Methods: A comprehensive search of MEDLINE, EMBASE, Web of Knowledge, Educational Resources Information Centre, Cochrane Central Register of Controlled Trials, PsycINFO, and Cumulative Index to Nursing and Allied Health Literature was conducted from 1990 to August 2017. Randomized controlled trials (RCTs) and cluster RCTs were eligible for inclusion. Two reviewers independently searched, screened, and assessed the study quality and extracted data. A meta-analysis was not deemed appropriate due to the heterogeneity of populations, interventions, comparisons, and outcomes. Therefore, a narrative synthesis is presented. Results: A total of 27 RCTs and 3 cluster RCTs with 3634 participants were included. Two studies evaluated gamification interventions, and the remaining evaluated serious gaming interventions. One study reported a small statistically significant difference between serious gaming and digital education of primary care physicians in the time to control blood pressure in a subgroup of their patients already taking antihypertensive medications. There was evidence of a moderate-to-large magnitude of effect from five studies evaluating individually delivered interventions for objectively measured knowledge compared with traditional learning. There was also evidence of a small-to-large magnitude of effect from 10 studies for improved skills compared with traditional learning. Two and four studies suggested equivalence between interventions and controls for knowledge and skills, respectively. Evidence suggested that serious gaming was at least as effective as other digital education modalities for these outcomes. There was insufficient evidence to conclude whether one type of serious gaming/gamification intervention is more effective than any other. There was limited evidence for the effects of serious gaming/gamification on professional attitudes. Serious gaming/gamification may improve satisfaction, but the evidence was limited. Evidence was of low or very low quality for all outcomes. Quality of evidence was downgraded due to the imprecision, inconsistency, and limitations of the study. Conclusions: Serious gaming/gamification appears to be at least as effective as controls, and in many studies, more effective for improving knowledge, skills, and satisfaction. However, the available evidence is mostly of low quality and calls for further rigorous, theory-driven research