Development and validation of the ACE tool: Assessing medical trainees' competency in evidence based medicine

BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis

Evidence based or person centered? An ontological debate

Evidence based medicine (EBM) is under critical debate, and person centered healthcare (PCH) has been proposed as an improvement. But is PCH offered as a supplement or as a replacement of EBM? Prima facie PCH only concerns the practice of medicine, while the contended features of EBM also include methods and medical model. I here argue that there are good philosophical reasons to see PCH as a radical alternative to the existing medical paradigm of EBM, since the two seem committed to conflicting ontologies. This paper aims to make explicit some of the most fundamental assumptions that motivate EBM and PCH, respectively, in order to show that the choice between them ultimately comes down to ontological preference. While EBM has a solid foundation in positivism, or what I here call Humeanism, PCH is more consistent with causal dispositionalism. I conclude that if there is a paradigmatic revolution on the way in medicine, it is first of all one of ontology

Moving from evidence-based medicine to evidence-based health.

While evidence-based medicine (EBM) has advanced medical practice, the health care system has been inconsistent in translating EBM into improvements in health. Disparities in health and health care play out through patients' limited ability to incorporate the advances of EBM into their daily lives. Assisting patients to self-manage their chronic conditions and paying attention to unhealthy community factors could be added to EBM to create a broader paradigm of evidence-based health. A perspective of evidence-based health may encourage physicians to consider their role in upstream efforts to combat socially patterned chronic disease

How are "teaching the teachers" courses in evidence based medicine evaluated? A systematic review

Background Teaching of evidence-based medicine (EBM) has become widespread in medical education. Teaching the teachers (TTT) courses address the increased teaching demand and the need to improve effectiveness of EBM teaching. We conducted a systematic review of assessment tools for EBM TTT courses. To summarise and appraise existing assessment methods for teaching the teachers courses in EBM by a systematic review. Methods We searched PubMed, BioMed, EmBase, Cochrane and Eric databases without language restrictions and included articles that assessed its participants. Study selection and data extraction were conducted independently by two reviewers. Results Of 1230 potentially relevant studies, five papers met the selection criteria. There were no specific assessment tools for evaluating effectiveness of EBM TTT courses. Some of the material available might be useful in initiating the development of such an assessment tool. Conclusion There is a need for the development of educationally sound assessment tools for teaching the teachers courses in EBM, without which it would be impossible to ascertain if such courses have the desired effect

GP trainees’ perceptions on learning EBM using conversations in the workplace : a video-stimulated interview study

Background To be able to practice evidence-based medicine (EBM) when making decisions for individual patients, it is important to learn how to combine the best available evidence with the patient's preferences and the physician's clinical expertise. In general practice training, these skills can be learned at the workplace using learning conversations: meetings between the supervising general practitioner (GP) and GP trainee to discuss medical practice, selected topics or professional performance. This study aimed to give insight into the perceptions of GP trainees on their EBM learning processes during learning conversations. Methods We held semi-structured video-stimulated elicitation interviews (n = 22) with GP trainees affiliated to GP training institutes in the Netherlands and Belgium. GP trainees were shown fragments of their learning conversations, enabling reflection during the interview. Taking an inductive approach, interview recordings were transcribed verbatim and analysed with NVivo software. Results GP trainees perceived learning conversations as useful for learning and discussing EBM. Multiple EBM learning activities were identified, such as discussing evidence together, relating evidence to cases in daily practice and discussing the supervisor's experience and the specific local context in the light of what the evidence recommends. However, for learning to occur, trainees need and expect specific behaviour, both from their supervisors and themselves. Supervisors should supply well-substantiated answers that are applicable in practice and give the trainee confirmation. In turn, the trainee needs to prepare well in order to ask focused, in-depth questions. A safe space allowing equal and open discussion between trainee and supervisor is perceived as an essential context for optimal EBM learning. Conclusions Our findings show that trainees find learning conversations useful for EBM learning in general practice. To bring EBM learning to its full potential, attention should be paid to optimising the behavioural and contextual factors found relevant to enhancing EBM learning

Teaching and learning evidence-based medicine: cross-sectional survey investigating knowledge and attitudes of teachers and learners in primary and secondary care

Evidence-based medicine (EBM) is an important component of quality healthcare and a key part of the curriculum for doctors in training. There have been no previous studies comparing attitudes and knowledge of doctors in primary and secondary care towards EBM practice and teaching and this study sets out to investigate this area. We asked participants, a stratified sample of general practitioners, hospital consultants, GP registrars and junior hospital doctors in Leicester, Northamptonshire and Rutland, UK, to complete a self-administered survey questionnaire and written knowledge test which provided ‘positive to evidence based practice’ (PEP) attitude scores and Manchester Short EBM Questionnaire Education for Primary Care (2007) 18: 45–57 # 2007 Radcliffe Publishing Limited WHAT IS ALREADY KNOWN IN THIS AREA. There is little evidence on the relationship between attitudes and knowledge in relation to evidence-based medicine (EBM) in family doctors, consultants and doctors intraining. WHAT THIS WORK ADDS. This study showed that, although general practitioners and general practitioner trainers were significantly less positive in attitude to EBM compared to GP registrars, junior hospital doctors and consultant respondents, they had significantly higher knowledge scores. This study demonstrated that the attitude (PEP) score and knowledge questionnaire(MANSEBMQ) have high reliability but require further research to demonstrate validity. SUGGESTIONS FOR FURTHER RESEARCH. There remain opportunities for refinement of the MANSEBMQ, validation against existing tools and further application in a larger study, including assessment of EBM knowledge and skills, before and after an educational process, involving students in clinically relevant and integrated EBM learning. Keywords: attitudes, evidence-based practice, general practice registrars, general practitioners, hospital doctors, primary care, secondary care(MANSEBMQ) knowledge scores of participants. The response rate was low which may have led to volunteer bias but there were sufficient responses to explore attitude scores and knowledge scores. Attitude(PEP) scores were highest in hospital consultants, intermediate in doctors in training and lowest in general practitioner (GP)respondents (mean score 71.7 vs 70.5 vs 67.2; P = 0.006). PEP scores were also highest in respondents with higher degrees (MD, PhD, MSc), intermediate in those with higher professional qualifications (MRCP, FRCS, MRCGP or equivalent) and lowest in those with none of these (mean score 72.9 vs 70.6 vs 67.2; P = 0.005). PEP scores were significantly higher(P = 0.002) in respondents who taught EBM (mean score 71.7, 95% CI 70.3 to 73.2, n=109, missing=5) compared with those who did not (mean score 68.6, 95% CI 67.3 to 69.9, n = 105, missing = 12) and in respondents with research experience (P < 0.001), research training (P < 0.001) and training in EBM (P = 0.001). There was a positive correlation between PEP score and MANSEBMQ score (P = 0.013). In contrast, and paradoxically opposite to the pattern of attitudes, knowledge scores were highest in GPs, intermediate in junior hospital doctors and lowest in consultant respondents (mean score 63.5 vs 61.9 vs 54.5, P=0.005). Although GPs and GP trainers were significantly less positive in attitude to EBM compared to GP registrars, junior hospital doctors and consultant respondents, they had significantly higher knowledge scores. This study demonstrated that the attitude(PEP) score and knowledge questionnaire (MANSEBMQ) have good reliability but require further research to demonstrate validity

Information-theoretic classification of SNOMED improves the organization of context-sensitive excerpts from Cochrane Reviews

The emphasis on evidence based medicine (EBM) has placed increased focus on finding timely answers to clinical questions in presence of patients. Using a combination of natural language processing for the generation of clinical excerpts and information theoretic distance based clustering, we evaluated multiple approaches for the efficient presentation of context-sensitive EBM excerpts