Development and validation of the ACE tool: Assessing medical trainees' competency in evidence based medicine

BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis

Satisfação materna com o cuidado da enfermeira materno-infantil em Campeche, México

OBJECTIVE: Evaluate and compare maternal-satisfaction (global and areas) with maternal-child nursing care (MSMINC) and to explore the relationship of MSMINC with wait time, length of visit, and maternal age and education. METHODS: Cross-sectional descriptive study comprising 213 mothers. Group 1 (n = 84), mothers of children agedEl objetivo de este estudio fue evaluar y comparar la satisfacción materna (global/áreas) con el cuidado de la enfermera materno infantil (MSMINC) y explorar la relación de MSMINC con el tiempo de espera, duración de la visita, edad y educación materna. Se trata de un estudio descriptivo transversal. Participaron 213 madres. Grupo 1, n = 84 madres de niñosOBJETIVO: Avaliar e comparar a satisfação materna (global e áreas) com o cuidado da enfermeira materno-infantil (SMAEMI) e explorar a relação da SMAEMI com o tempo de espera e duração da visita, idade e educação da mãe. MÉTODOS: ESTUdo descritivo-transversal com a participação de 213 mães. Grupo 1, n = 84 mães de criança

The Rourke Baby Record Infant/Child Maintenance Guide: do doctors use it, do they find it useful, and does using it improve their well-baby visit records?

Background. The Rourke Baby Record (RBR) - http://www.rourkebabyrecord.ca - is a freely available evidence-based structured form for child health surveillance from zero to five years. Family physicians/general practitioners (FP/GPs) doing office based well-baby care in three Ontario Canada cities (London, Ottawa, and Toronto) were randomly sampled to study the prevalence and utility of the RBR and documentation of well-baby visits. Methods. Database with telephone confirmation was conducted to assess the prevalence of use of the RBR. Study Part 1: Questionnaire mailed to a random sample of 100 RBR users. Outcome measures were utility of, helpfulness of, and suggestions for the RBR. Descriptive analysis was employed. Study Part 2: Retrospective chart review of well-baby visits by 38 FP/GPs using student t-tests and factor analysis. Outcome measures were well-baby visit documentation of growth, nutrition, safety issues, developmental milestones, physical examination, and overall comprehensiveness. Results. The RBR was used by 78.5% (402/512) of successfully contacted FP/GPs who did well-baby care in these 3 cities. Study Part 1: Questionnaire respondents (N = 41/100) used the RBR in several ways, and found it most helpful for assessing healthy child development, charting/recording the visits, managing time effectively, addressing parent concerns, identifying health problems, and identifying high risk situations. The RBR was seen to be least helpful as a tool for managing or for referring identified health problems. Study Part 2: Charts from a total of 1,378 well-baby visits on 176 children were audited. Well-baby care provided by the 20 FP/GPs who used the RBR compared to that by the 18 non-users was statistically more likely to include documentation of type of feeding (p = 0.023), discussion of safety issues (p < 0.001), assessment of development (p = 0.001), and overall comprehensiveness (p < 0.001). Well-baby care provided by the RBR users compared to that by the non-users was not more likely to include documentation of measurement of growth (p = 0.097), or physical examination (p = 0.828). Conclusion. The RBR was widely used by FP/GPs in these settings. RBR users found it helpful for many purposes, and had a consistently high rate of documentation of many aspects of well-baby care. The Rourke Baby Record has become a de facto gold standard clinical practice tool in knowledge translation for pediatric preventive medicine and health surveillance for primary care pediatric providers

Validation of the modified Fresno Test: assessing physical therapists' evidence based practice knowledge and skills

<p>Abstract</p> <p>Background</p> <p>Health care educators need valid and reliable tools to assess evidence based practice (EBP) knowledge and skills. Such instruments have yet to be developed for use among physical therapists. The Fresno Test (FT) has been validated only among general practitioners and occupational therapists and does not assess integration of research evidence with patient perspectives and clinical expertise. The purpose of this study was to develop and validate a modified FT to assess EBP knowledge and skills relevant to physical therapist (PT) practice.</p> <p>Methods</p> <p>The FT was modified to include PT-specific content and two new questions to assess integration of patient perspectives and clinical expertise with research evidence. An expert panel reviewed the test for content validity. A cross-sectional cohort representing three training levels (EBP-novice students, EBP-trained students, EBP-expert faculty) completed the test. Two blinded raters, not involved in test development, independently scored each test. Construct validity was assessed through analysis of variance for linear trends among known groups. Inter and intra-rater reliability, internal consistency, item discrimination index, item total correlation, and difficulty were analyzed.</p> <p>Results</p> <p>Among 108 participants (31 EBP-novice students, 50 EBP-trained students, and 27 EBP-expert faculty), there was a statistically significant (p < 0.0001) difference in total score corresponding to training level. Total score reliability and psychometric properties of items modified for discipline-specific content were excellent [inter-rater (ICC (2,1)] = 0.91); intra-rater (ICC (2,1)] = 0.95, 0.96)]. Cronbach's α was 0.78. Of the two new items, only one had strong psychometric properties.</p> <p>Conclusions</p> <p>The 13-item modified FT presented here is a valid, reliable assessment of physical therapists' EBP knowledge and skills. One new item assesses integration of patient perspective as part of the EBP model. Educators and researchers may use the 13-item modified FT to evaluate PT EBP curricula and physical therapists' EBP knowledge and skills.</p

The value of health care – a matter of discussion in Germany

BACKGROUND: Interest in assessing the value of health-care services in Germany has considerably increased since the foundation of the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG (Institute for Quality and Efficiency in Health Care). The practical application of value assessment illustrates how problematic the process can be. In all decisions made for the provision of health care, data concerning the measurable dimensions (quantity and quality of efficacy and effectiveness, validity of the results and costs) flow into a complex and not yet standardized decision-making process concerning public financing. Some of these decisions are based on data of uncertain validity, unknown reproducibility and unclear appropriateness. DISCUSSION: In this paper we describe the theoretical aspects of value from psychological and economic viewpoints and discuss national and international approaches. Methodic details and difficulties in assessing the value of health-care services are analysed. A definition of the intangible value of health-care services will be proposed which contains only three factors: the absolute risk reduction (usually a measure of efficacy), the validity of the scientific papers examined and the type of the expected effectiveness (prevention of death and disability, restitution of well-being). The intangible value describes the additional benefit when comparing two possible actions, like treatment or observation only. CONCLUSION: The description of intangible value from the viewpoint of different stakeholders is a useful measure for subsequent steps (not discussed here) – the evaluation of costs and of patient benefit. A standardised, transparent, fair and democratic evaluation is essential for the definition of a basic benefit package

Effect of a web-based chronic disease management system on asthma control and health-related quality of life: study protocol for a randomized controlled trial

<p>Abstract</p> <p>Background</p> <p>Asthma is a prevalent and costly disease resulting in reduced quality of life for a large proportion of individuals. Effective patient self-management is critical for improving health outcomes. However, key aspects of self-management such as self-monitoring of behaviours and symptoms, coupled with regular feedback from the health care team, are rarely addressed or integrated into ongoing care. Health information technology (HIT) provides unique opportunities to facilitate this by providing a means for two way communication and exchange of information between the patient and care team, and access to their health information, presented in personalized ways that can alert them when there is a need for action. The objective of this study is to evaluate the acceptability and efficacy of using a web-based self-management system, My Asthma Portal (MAP), linked to a case-management system on asthma control, and asthma health-related quality of life.</p> <p>Methods</p> <p>The trial is a parallel multi-centered 2-arm pilot randomized controlled trial. Participants are randomly assigned to one of two conditions: a) MAP and usual care; or b) usual care alone. Individuals will be included if they are between 18 and 70, have a confirmed asthma diagnosis, and their asthma is classified as not well controlled by their physician. Asthma control will be evaluated by calculating the amount of fast acting beta agonists recorded as dispensed in the provincial drug database, and asthma quality of life using the Mini Asthma Related Quality of Life Questionnaire. Power calculations indicated a needed total sample size of 80 subjects. Data are collected at baseline, 3, 6, and 9 months post randomization. Recruitment started in March 2010 and the inclusion of patients in the trial in June 2010.</p> <p>Discussion</p> <p>Self-management support from the care team is critical for improving chronic disease outcomes. Given the high volume of patients and time constraints during clinical visits, primary care physicians have limited time to teach and reinforce use of proven self-management strategies. HIT has the potential to provide clinicians and a large number of patients with tools to support health behaviour change.</p> <p>Trial Registration</p> <p>Current Controlled Trials <a href="http://www.controlled-trials.com/ISRCTN34326236">ISRCTN34326236</a>.</p

Teaching evidence based medicine literature searching skills to medical students during the clinical years - a protocol for a randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>Two of the key steps in evidence based medicine (EBM) are being able to construct a clinical question and effectively search the literature to source relevant information. No evidence currently exists that informs whether such skills should be taught to medical students during their pre-clinical years, or delivered to include both the pre-clinical and clinical years of study. This is an important component of curriculum design as the level of clinical maturity of students can affect their perception of the importance and uptake of EBM principles in practice.</p> <p>Methods/Design</p> <p>A randomised controlled trial will be conducted to identify the effectiveness of delivering a formal workshop in EBM literature searching skills to third year medical students entering their clinical years of study. The primary outcome of EBM competency in literature searching skills will be evaluated using the Fresno tool.</p> <p>Discussion</p> <p>This trial will provide novel information on the effectiveness of delivering a formal education workshop in evidence based medicine literature searching skills during the clinical years of study. The result of this study will also identify the impact of teaching EBM literature searching skills to medical students during the clinical years of study.</p