Feasibility and impact of an evidence-based electronic decision support system for diabetes care in family medicine: protocol for a cluster randomized controlled trial

Background: In Belgium, the construction of the national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, healthcare providers, Evidence-Based Medicine (EBM) partners, and vendors of Electronic Health Records (EHR) is unique to this project. All Belgian healthcare professionals get free access to an up-to-date database of validated Belgian and nearly 1,000 international guidelines, incorporated in a portal that also provides EBM information from sources other than guidelines, including computerized clinical decision support that is integrated in the EHRs. The EBMeDS system is the electronic evidence-based decision support system of EBMPracticeNet. The EBMeDS system covers all clinical areas of diseases and could play a crucial role in response to the emerging challenge posed by chronic conditions. Diabetes was chosen as the analysis topic of interest. The objective of this study is to assess the effectiveness of EBMeDS use in improving diabetes care. This objective will be enhanced by a formal process evaluation to provide crucial information on the feasibility of using the system in daily Belgian family medicine. Methods: The study is a cluster-randomized trial with before/after measurements conducted in Belgian family medicine. Physicians' practices will be randomly assigned to the intervention or control group in a 1: 1 ratio, to receive either the EBMeDS reminders or to follow the usual care process. Randomization will be performed by a statistical consultant with an electronic random list generator, anonymously for the researchers. The follow-up period of the study will be 12 months with interim analysis points at 3, 6 and 9 months. Primary outcome is the one-year pre- to post-implementation change in HbA1c. Patients will not be informed about the intervention. Data analysts will be kept blinded to the allocation. Discussion: The knowledge obtained in this study will be useful for further integration in other Belgian software packages. Users' perceptions and process evaluation will provide information for improving the feasibility of the system

The attitude of Belgian social insurance physicians towards evidence-based practice and clinical practice guidelines

<p>Abstract</p> <p>Background</p> <p>Evidence-based medicine has broadened its scope and is starting to reach insurance medicine. Although still in its initial stages, physicians in the area of insurance medicine should keep up-to-date with the evidence on various diseases in order to correctly assess disability and to give appropriate advice about health care reimbursement. In order to explore future opportunities of evidence-based medicine to improve daily insurance medicine, there is a need for qualitative studies to better understand insurance physicians' perceptions of EBM. The present study was designed to identify the attitude of insurance physicians towards evidence-based medicine and clinical practice guidelines, and to determine their ability to access, retrieve and appraise the health evidence and the barriers for applying evidence to practice.</p> <p>Methods</p> <p>A cross-sectional survey study was carried out among all Dutch-speaking insurance physicians employed at one of the six Belgian social insurance sickness funds and at the National Institute of Disability and Health care Insurance (n = 224). Chi-square tests were used to compare nominal and ordinal variables. Student's t-tests, ANOVA, Mann-Whitney and Kruskal-Wallis were used to compare means of continuous variables for different groups.</p> <p>Results</p> <p>The response rate was 48.7%. The majority of respondents were positive towards evidence-based medicine and clinical practice guidelines. Their knowledge of EBM was rather poor. Perceived barriers for applying evidence to practice were mainly time and lack of EBM skills.</p> <p>Conclusion</p> <p>Although the majority of physicians were positive towards EBM and welcomed more guidelines, the use of evidence and clinical practice guidelines in insurance medicine is low at present. It is in the first place important to eradicate the perceived inertia which limits the use of EBM and to further investigate the EBM principles in the context of insurance medicine. Available high-quality evidence-based resources (at the moment mainly originating from other medical fields) need to be structured in a way that is useful for insurance physicians and global access to this information needs to be ensured.</p

Effectiveness of electronic guideline-based implementation systems in ambulatory care settings - a systematic review

<p>Abstract</p> <p>Background</p> <p>Electronic guideline-based decision support systems have been suggested to successfully deliver the knowledge embedded in clinical practice guidelines. A number of studies have already shown positive findings for decision support systems such as drug-dosing systems and computer-generated reminder systems for preventive care services.</p> <p>Methods</p> <p>A systematic literature search (1990 to December 2008) of the English literature indexed in the Medline database, Embase, the Cochrane Central Register of Controlled Trials, and CRD (DARE, HTA and NHS EED databases) was conducted to identify evaluation studies of electronic multi-step guideline implementation systems in ambulatory care settings. Important inclusion criterions were the multidimensionality of the guideline (the guideline needed to consist of several aspects or steps) and real-time interaction with the system during consultation. Clinical decision support systems such as one-time reminders for preventive care for which positive findings were shown in earlier reviews were excluded. Two comparisons were considered: electronic multidimensional guidelines versus usual care (comparison one) and electronic multidimensional guidelines versus other guideline implementation methods (comparison two).</p> <p>Results</p> <p>Twenty-seven publications were selected for analysis in this systematic review. Most designs were cluster randomized controlled trials investigating process outcomes more than patient outcomes. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group (comparison one). No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes (comparison two).</p> <p>Conclusions</p> <p>There is little evidence at the moment for the effectiveness of an increasingly used and commercialised instrument such as electronic multidimensional guidelines. After more than a decade of development of numerous electronic systems, research on the most effective implementation strategy for this kind of guideline-based decision support systems is still lacking. This conclusion implies a considerable risk towards inappropriate investments in ineffective implementation interventions and in suboptimal care.</p

Evidence-Based Health Care: barriers and facilitators of electronic guideline implementation and development in ambulatory settings

IntroductionTheoretically, doing what has been proven to be effective or stopping was is not working has the potential to provide health benefits or to reduce patient risks and waste of health care resources. As a consequence, it is essential that we find ways of implementing this evidence into practice. One possible way to implement evidence into practice is the use of instruments such as clinical practice guidelines (CPGs).Giving the broad spectrum of chronic illnesses treated in ambulatory care, the potential benefits of well-developed and effectively implemented CPGs and the various opportunities of electronic approaches, the objective of this thesis was to study the barriers and facilitators of electronic guideline implementation in ambulatory settings (part I) and to evaluate an innovative electronic development method (part II) for clinical practice guidelines.MethodsFor part I, we first performed a systematic review to synthesize current evidence on the effectiveness of computer-based guideline implementation systems in ambulatory care settings. A survey study was carried out to explore future opportunities of evidence-based practice guidelines implementation in occupational health and insurance medicine. We then evaluated family physicians acceptance of an electronic clinical decision support system (the EBMeDS system) in routine practice.For part II, we developed and evaluated a new consensus method based on human computation techniques. The human computation method is comparable with an online Delphi approach packaged as a game. The method has been proven to be effective in building large knowledge bases. Differences between the human computation method and an informal consensus method were measured by means of a randomized controlled trial in a population of students following a master of nursing and obstetrics and a population of trainees in family medicine. Primary study outcomes were differences in the changes in evidence scores and in agreement across multiple rounds for four clinical scenarios of low back pain.ResultsTwenty-seven studies were selected for analysis in our systematic review. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group. No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes.Our qualitative research showed that the majority of occupational health and insurance medicine physicians had a positive attitude towards evidence-based medicine and clinical practice guidelines. However, there were major barriers in the legislative framework and the information infrastructure that did not allow successful implementation within the given time-frame. Therefore, a shift to family medicine was made, where a new evidence-based decision support system, EBMeDS, was in its implementation phase.The respondents of our EBMeDS-study demonstrated a relatively high degree of acceptance. Mean intention to keep using the system was high. Their perception of the ease of use of the system, usefulness and the facilitating conditions was in general positive. Part of the statements gave mixed results and could be identified as important points of interest for future implementation initiatives and system improvements. In our RCT of guideline development, we demonstrated that our innovative method based on human computation may be a time efficient and acceptable method for guideline development specifically for the type of scenarios where the evidence shows no resonance with participants beliefs. When clear evidence was available and when some controversy about the evidence existed, changes in answers across rounds were more evidence-based in the HC groups compared to the IC groups. The human computation method was able to create a consensus to the same degree as an informal face-to-face method. Because it often takes years to achieve general acceptance of evidence by the medical profession, the HC method could also be very useful when new evidence emerges and existing guidelines need to be updated. When no evidence was available or when the evidence supported participants beliefs or usual practice, a higher degree of agreement was reached in the IC groups compared to the HC groups. Arguments used in the decision making process of the informal consensus method were mainly based on personal conviction or preferences. Very few evidence arguments were quoted during informal group discussions (7% in the population of students following a master of nursing and obstetrics versus 1% in the population of trainees in family medicine).DiscussionA number of studies have shown positive findings for electronic decision support systems in areas such as drug-dosing systems and computer-based reminder systems for e.g. preventive services. However, -although increasingly used and commercialised- little evidence exists at the moment for more complex electronic guideline implementation. This thesis made a significant contribution to the understanding of electronic guideline implementation barriers from a real-life perspective. Guideline recommendations with a high degree of certainty and agreement will more easily be implemented. More complex guidelines will not pass in a spontaneous way and calls for adaptive behaviors as well for the development as for the implementation process. We were able to demonstrate that the choice of the consensus method in guideline development might have an important influence on the guideline content. Our new method of human computation proved very useful in the introduction of clinical evidence arguments, while neutralising for psychosocial influences by authoritarian opinions.status: publishe

Family Physicians' Perceptions and Use of Electronic Clinical Decision Support During the First Year of Implementation

An electronic decision support system (the EBMeDS system) was integrated in one of the Electronic Medical Records (EMR) of Belgian family physicians (Feb 2010). User acceptance of the system is considered as a necessary condition for the effective implementation of any IT project. Facilitators, barriers and issues of non-acceptance need to be understood in view of a successful implementation and to minimize unexpected adoption behavior. Objectives of the study were the assessment of users' perceptions towards the recently implemented EBMeDS system, the investigation of user-interactions with the system and possible relationships between perceptions and use. A mixed evaluation approach was performed consisting of a qualitative and a quantitative analysis. The technology acceptance model of UTAUT was used as a structural model for the development of our questionnaire to identify factors that may account for acceptance and use of the EBMeDS system (seven-point Likert scales). A quantitative analysis of computer-recorded user interactions with the system was performed for an evaluation period of 3 months to assess the actual use of the system. Qualitative and quantitative analysis were linked to each other. Thirty-nine family physicians (12 %) completed the survey. The majority of respondents (66 %) had a positive attitude towards the system in general. Mean intention to keep using the system was high (5,91 ± 1,33). Their perception of the ease of use of the system (mean 5,04 ± 1,41), usefulness (mean 4,69 ± 1,35) and facilitating conditions (4,43 ± 1,13) was in general positive. Only 0,35 % of reminders were requested on demand, the other 99,62 % of reminders displayed automatically. Detailed guidelines (long) were requested for 0,47 % of reminders automatically shown versus 16,17 % of reminders on request. The script behind the reminders was requested for 8,4 % of reminders automatically shown versus 13,6 % of reminders on request. The majority of respondents demonstrated a relatively high degree of acceptance towards the EBMeDS system. Although the majority of respondents was in general positive towards the ease of use of the system, usefulness and facilitating conditions, part of the statements gave rather mixed results and could be identified as important points of interest for future implementation initiatives and system improvements. It has to be stressed that our population consisted of a convenience sample of early adopters, willing to answer a questionnaire. The willingness to adopt the system depends on the willingness to use ICPC coding. As such, the quality of reminding partly depends on the quality of coding. There is a need to reach a larger population of physicians (including physicians who never used the system or stopped using the system) to validate the results of this survey.status: publishe

Feasibility and impact of an evidence-based electronic decision support system for diabetes care in family medicine: protocol for a cluster randomized controlled trial

In Belgium, the construction of the national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, healthcare providers, Evidence-Based Medicine (EBM) partners, and vendors of Electronic Health Records (EHR) is unique to this project. All Belgian healthcare professionals get free access to an up-to-date database of validated Belgian and nearly 1,000 international guidelines, incorporated in a portal that also provides EBM information from sources other than guidelines, including computerized clinical decision support that is integrated in the EHRs.The EBMeDS system is the electronic evidence-based decision support system of EBMPracticeNet. The EBMeDS system covers all clinical areas of diseases and could play a crucial role in response to the emerging challenge posed by chronic conditions. Diabetes was chosen as the analysis topic of interest. The objective of this study is to assess the effectiveness of EBMeDS use in improving diabetes care. This objective will be enhanced by a formal process evaluation to provide crucial information on the feasibility of using the system in daily Belgian family medicine.status: publishe

Human computation as a new method for evidence-based knowledge transfer in web-based guideline development groups: A proof of concept randomized controlled trial

BACKGROUND: Guideline developers use different consensus methods to develop evidence-based clinical practice guidelines. Previous research suggests that existing guideline development techniques are subject to methodological problems and are logistically demanding. Guideline developers welcome new methods that facilitate a methodologically sound decision-making process. Systems that aggregate knowledge while participants play a game are one class of human computation applications. Researchers have already proven that these games with a purpose are effective in building common sense knowledge databases. OBJECTIVE: We aimed to evaluate the feasibility of a new consensus method based on human computation techniques compared to an informal face-to-face consensus method. METHODS: We set up a randomized design to study 2 different methods for guideline development within a group of advanced students completing a master of nursing and obstetrics. Students who participated in the trial were enrolled in an evidence-based health care course. We compared the Web-based method of human-based computation (HC) with an informal face-to-face consensus method (IC). We used 4 clinical scenarios of lower back pain as the subject of the consensus process. These scenarios concerned the following topics: (1) medical imaging, (2) therapeutic options, (3) drugs use, and (4) sick leave. Outcomes were expressed as the amount of group (dis)agreement and the concordance of answers with clinical evidence. We estimated within-group and between-group effect sizes by calculating Cohen's d. We calculated within-group effect sizes as the absolute difference between the outcome value at round 3 and the baseline outcome value, divided by the pooled standard deviation. We calculated between-group effect sizes as the absolute difference between the mean change in outcome value across rounds in HC and the mean change in outcome value across rounds in IC, divided by the pooled standard deviation. We analyzed statistical significance of within-group changes between round 1 and round 3 using the Wilcoxon signed rank test. We assessed the differences between the HC and IC groups using Mann-Whitney U tests. We used a Bonferroni adjusted alpha level of .025 in all statistical tests. We performed a thematic analysis to explore participants' arguments during group discussion. Participants completed a satisfaction survey at the end of the consensus process. RESULTS: Of the 135 students completing a master of nursing and obstetrics, 120 participated in the experiment. We formed 8 HC groups (n=64) and 7 IC groups (n=56). The between-group comparison demonstrated that the human computation groups obtained a greater improvement in evidence scores compared to the IC groups, although the difference was not statistically significant. The between-group effect size was 0.56 (P=.30) for the medical imaging scenario, 0.07 (P=.97) for the therapeutic options scenario, and 0.89 (P=.11) for the drug use scenario. We found no significant differences in improvement in the degree of agreement between HC and IC groups. Between-group comparisons revealed that the HC groups showed greater improvement in degree of agreement for the medical imaging scenario (d=0.46, P=.37) and the drug use scenario (d=0.31, P=.59). Very few evidence arguments (6%) were quoted during informal group discussions. CONCLUSIONS: Overall, the use of the IC method was appropriate as long as the evidence supported participants' beliefs or usual practice, or when the availability of the evidence was sparse. However, when some controversy about the evidence existed, the HC method outperformed the IC method. The findings of our study illustrate the importance of the choice of the consensus method in guideline development. Human computation could be an acceptable methodology for guideline development specifically for scenarios in which the evidence shows no resonance with participants' beliefs. Future research is needed to confirm the results of this study and to establish practical significance in a controlled setting of multidisciplinary guideline panels during real-life guideline development.status: publishe

Rigour of development does not AGREE with recommendations in practice guidelines on the use of ice for acute ankle sprains

Objective This study evaluated whether the Appraisal of Guidelines Research and Evaluation (AGREE) rigour of development score of practice guidelines on ice for acute ankle sprains is related to the convergence between recommendations. Design The authors systematically reviewed guidelines on ice for acute ankle sprains. Four appraisers independently used the AGREE instrument to evaluate the rigour of development of selected guidelines. For each guideline, one reviewer listed the cited evidence on ice and calculated a cited evidence score. The authors plotted the recommended durations and numbers of ice applications over the standardised rigour of development score to explore the relationships. Data sources Three reviewers searched for guidelines in Medline, Embase, Sportdiscus, PEDro, G-I-N Guideline Library, Trip Database, SumSearch, National Guideline Clearinghouse and the Health Technology Assessment database, and conducted a web-based search for guideline development organisations. Eligibility criteria Eligible guidelines had a development methodology that included a process to search or use results from scientific studies and the participation of an expert group to formulate recommendations. Results The authors identified 21 guidelines, containing clinically significant variations in recommended durations and numbers of ice applications. The median standardised rigour of development score was 57% (IQR 18 to 77). Variations occurred evenly among guidelines with low moderate or high rigour scores. The median evidence citation score in the guidelines was 7% (IQR 0 to 61). Conclusions There is no relationship between the rigour of development score and the recommendations in guidelines on ice for acute ankle sprains. The guidelines suffered from methodological problems which were not captured by the AGREE instrument.status: publishe

Effectiveness of nonresuscitative first aid training in laypersons: a systematic review

STUDY OBJECTIVE: This study reviewed evidence on the effects of nonresuscitative first aid training on competence and helping behavior in laypersons. METHODS: We identified randomized and nonrandomized controlled trials and interrupted time series on nonresuscitative first aid training for laypersons by using 12 databases (including MEDLINE, EMBASE, and PsycINFO), hand searching, reference checking, and author communication. Two reviewers independently evaluated selected studies with the Cochrane Effective Practice and Organisation of Care Review Group quality criteria. One reviewer extracted data with a standard form and another checked them. In anticipation of substantial heterogeneity across studies, we elected a descriptive summary of the included studies. RESULTS: We included 4 studies, 3 of which were randomized trials. We excluded 11 studies on quality issues. Two studies revealed that participants trained in first aid demonstrated higher written test scores than controls (poisoning first aid: relative risk 2.11, 95% confidence interval [CI] 1.64 to 2.72; various first aid cases: mean difference 4.75, 95% CI 3.02 to 6.48). Two studies evaluated helping responses during unannounced simulations. First aid training improved the quality of help for a bleeding emergency (relative risk 25.94; 95% CI 3.60 to 186.93), not the rate of helping (relative risk 1.13; 95% CI 0.88 to 1.45). Training in first aid and helping behavior increased the helping rates in a chest pain emergency compared with training in first aid only (relative risk 2.80; 95% CI 1.05 to 7.50) or controls (relative risk 3.81; 95% CI 0.98 to 14.89). Participants trained in first aid only did not help more than controls (relative risk 1.36; 95% CI 0.28 to 6.61). CONCLUSION: First aid programs that also train participants to overcome inhibitors of emergency helping behavior could lead to better help and higher helping rates.status: publishe

Computerized clinical decision support system for diabetes in primary care does not improve quality of care: a cluster-randomized controlled trial

BACKGROUND: The EBMeDS system is the computerized clinical decision support (CCDS) system of EBPNet, a national computerized point-of-care information service in Belgium. There is no clear evidence of more complex CCDS systems to manage chronic diseases in primary care practices (PCPs). The objective of this study was to assess the effectiveness of EBMeDS use in improving diabetes care. METHODS: A cluster-randomized trial with before-and-after measurements was performed in Belgian PCPs over 1 year, from May 2017 to May 2018. We randomly assigned 51 practices to either the intervention group (IG), to receive the EBMeDS system, or to the control group (CG), to receive usual care. Primary and secondary outcomes were the 1-year pre- to post-implementation change in HbA1c, LDL cholesterol, and systolic and diastolic blood pressure. Composite patient and process scores were calculated. A process evaluation was added to the analysis. Results were analyzed at 6 and 12 months. Linear mixed models and logistic regression models based on generalized estimating equations were used where appropriate. RESULTS: Of the 51 PCPs that were enrolled and randomly assigned (26 PCPs in the CG and 25 in the IG), 29 practices (3815 patients) were analyzed in the study: 2464 patients in the CG and 1351 patients in the IG. No change differences existed between groups in primary or secondary outcomes. Change difference between CG and IG after 1-year follow-up was - 0.09 (95% CI - 0.18; 0.01, p-value = 0.06) for HbA1c; 1.76 (95% CI - 0.46; 3.98, p-value = 0.12) for LDL cholesterol; and 0.13 (95% CI - 0.91; 1.16, p-value = 0.81) and 0.12 (95% CI - 1.25;1.49, p-value = 0.86) for systolic and diastolic blood pressure respectively. The odds ratio of the IG versus the CG for the probability of no worsening and improvement was 1.09 (95% CI 0.73; 1.63, p-value = 0.67) for the process composite score and 0.74 (95% CI 0.49; 1.12, p-value = 0.16) for the composite patient score. All but one physician was satisfied with the EBMeDS system. CONCLUSIONS: The CCDS system EBMeDS did not improve diabetes care in Belgian primary care. The lack of improvement was mainly caused by imperfections in the organizational context of Belgian primary care for chronic disease management and shortcomings in the system requirements for the correct use of the EBMeDS system (e.g., complete structured records). These shortcomings probably caused low-use rates of the system. TRIAL REGISTRATION: ClinicalTrials.gov, NCT01830569, Registered 12 April 2013.status: publishe