The mCME Project: A Randomized Controlled Trial of an SMS-Based Continuing Medical Education Intervention for Improving Medical Knowledge among Vietnamese Community Based Physicians’ Assistants

<div><p>Background</p><p>Community health workers (CHWs) provide critical services to underserved populations in low and middle-income countries, but maintaining CHW’s clinical knowledge through formal continuing medical education (CME) activities is challenging and rarely occurs. We tested whether a Short Message Service (SMS)-based mobile CME (mCME) intervention could improve medical knowledge among a cadre of Vietnamese CHWs (Community Based Physician’s Assistants–CBPAs) who are the leading providers of primary medical care for rural underserved populations.</p><p>Methods</p><p>The mCME Project was a three arm randomized controlled trial. Group 1 served as controls while Groups 2 and 3 experienced two models of the mCME intervention. Group 2 (passive model) participants received a daily SMS bullet point, and were required to reply to the text to acknowledge receipt; Group 3 (interactive model) participants received an SMS in multiple choice question format addressing the same thematic area as Group 2, entering an answer (A, B, C or D) in their response. The server provided feedback immediately informing the participant whether the answer was correct. Effectiveness was based on standardized examination scores measured at baseline and endline (six months later). Secondary outcomes included job satisfaction and self-efficacy.</p><p>Results</p><p>638 CBPAs were enrolled, randomized, and tested at baseline, with 592 returning at endline (93.7%). Baseline scores were similar across all three groups. Over the next six months, participation of Groups 2 and 3 remained high; they responded to >75% of messages. Group 3 participants answered 43% of the daily SMS questions correctly, but their performance did not improve over time. At endline, the CBPAs reported high satisfaction with the mCME intervention, and deemed the SMS messages highly relevant. However, endline exam scores did not increase over baseline, and did not differ between the three groups. Job satisfaction and self-efficacy scores also did not improve. Average times spent on self-study per week did not increase, and the kinds of knowledge resources used by the CBPAs did not differ between the three groups; textbooks, while widely available, were seldom used.</p><p>Conclusions</p><p>The SMS-based mCME intervention, while feasible and acceptable, did not result in increased medical knowledge. We hypothesize that this was because the intervention failed to stimulate lateral learning. For an intervention of this kind to be effective, it will be essential to find more effective ways to couple SMS as a stimulus to promote increased self-study behaviors.</p><p>Trial Registration</p><p>ClinicalTrials.gov <a href="https://clinicaltrials.gov/ct2/show/NCT02381743" target="_blank">NCT02381743</a></p></div

Weekly non-conforming answer rates (Group 3 only).

<p>This summarizes the week-by-week average rates of non-conforming answers sent back by Group 3 participants. Responses were interpretable only if in the formats ‘a’, ‘b’, ‘c’, ‘d’, or ‘A’, ‘B’, ‘C’, or ‘D’. All other responses were ‘non-conforming’. Non-conforming answers were most common in the initial few weeks of the study, and then fell to < 10%.</p

Conceptual model for how the mCME intervention was hypothesized to work to improve medical knowledge.

<p>The figure outlines our model for the mCME intervention. The figure outlines two pathways that could lead to our desired output: improvement on the endline exam score. The first is learning from the information within the SMS messages themselves (the weak pathway). However, these are very brief, cover each topic superficially, and so would not be expected to have much effect per se. Rather, the SMS was hoped to serve as a stimulus promoting increased self-study on the same thematic areas as addressed in the messages through lateral learning (the strong pathway).</p

Impact of mCME intervention on key secondary endpoints: self-efficacy and job satisfaction.

<p>Impact of mCME intervention on key secondary endpoints: self-efficacy and job satisfaction.</p

Weekly correct answer rates (Group 3 only).

<p>The figure summarizes the proportion of Group 3 participants who provided a correct response to the daily SMS question over the 6 months of the study. While there was considerable week-to-week variation, there was no trend to improvement over time.</p

Study flow diagram.

<p>Study flow diagram.</p

Seating diagram for baseline and endline evaluations.

<p>Each cell represents an individual sitting a desk taking a specified version and sub-version of the examination. Each 4x4 block of sixteen seats can be repeated ad infinitum, and will never result in two individuals who are taking the same exam versions sitting adjacent to each other in any direction. This was intended to minimize potential that participants could work together or share answers during the baseline and endline evaluations. It can flexibly be adapted to suit different sized/shaped rooms. <b>E1V1</b> = Exam version 1, sub-version 1; <b>E1V2</b> = Exam version 1, sub-version 2. <b>E2V1</b> = Exam version 2, sub-version 1; <b>E2V2</b> = Exam version 2, sub-version 2.</p

Impact of mCME intervention on endline self-education behaviors.

<p>Impact of mCME intervention on endline self-education behaviors.</p

Weekly response rates (Groups 2 and 3).

<p>Group 2 and 3 participants were required to respond to the daily SMS messages. The figure summarizes the weekly average response rates for the two groups separately. As can be seen, participation was high for both groups, but statistically significantly higher for Group 3 participants. Group 1 participants were not asked to respond to the messages.</p