Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review.

Review date: 1969 to 2003, 34 years. Background and context: Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus. Objectives: Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'. Search strategy: The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the 'grey literature' were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality. Inclusion and exclusion criteria: Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention. Data extraction: Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol. Data synthesis: Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis. Headline results: Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following: providing feedback-51 (47%) journal articles reported that educational feedback is the most important feature of simulation-based medical education; repetitive practice-43 (39%) journal articles identified repetitive practice as a key feature involving the use of high-fidelity simulations in medical education; curriculum integration-27 (25%) journal articles cited integration of simulation-based exercises into the standard medical school or postgraduate educational curriculum as an essential feature of their effective use; range of difficulty level-15 (14%) journal articles address the importance of the range of task difficulty level as an important variable in simulation-based medical education; multiple learning strategies-11 (10%) journal articles identified the adaptability of high-fidelity simulations to multiple learning strategies as an important factor in their educational effectiveness; capture clinical variation-11 (10%) journal articles cited simulators that capture a wide variety of clinical conditions as more useful than those with a narrow range; controlled environment-10 (9%) journal articles emphasized the importance of using high-fidelity simulations in a controlled environment where learners can make, detect and correct errors without adverse consequences; individualized learning-10 (9%) journal articles highlighted the importance of having reproducible, standardized educational experiences where learners are active participants, not passive bystanders; defined outcomes-seven (6%) journal articles cited the importance of having clearly stated goals with tangible outcome measures that will more likely lead to learners mastering skills; simulator validity-four (3%) journal articles provided evidence for the direct correlation of simulation validity with effective learning. Conclusions: While research in this field needs improvement in terms of rigor and quality, high-fidelity medical simulations are educationally effective and simulation-based education complements medical education in patient care settings

A critical review of simulation-based medical education research: 2003-2009

This article reviews and critically evaluates historical and contemporary research on simulation-based medical education (SBME). It also presents and discusses 12 features and best practices of SBME that teachers should know in order to use medical simulation technology to maximum educational benefit. This qualitative synthesis of SBME research and scholarship was carried out in two stages. Firstly, we summarised the results of three SBME research reviews covering the years 1969-2003. Secondly, we performed a selective, critical review of SBME research and scholarship published during 2003-2009. The historical and contemporary research synthesis is reported to inform the medical education community about 12 features and best practices of SBME: (i) feedback; (ii) deliberate practice; (iii) curriculum integration; (iv) outcome measurement; (v) simulation fidelity; (vi) skill acquisition and maintenance; (vii) mastery learning; (viii) transfer to practice; (ix) team training; (x) high-stakes testing; (xi) instructor training, and (xii) educational and professional context. Each of these is discussed in the light of available evidence. The scientific quality of contemporary SBME research is much improved compared with the historical record. Development of and research into SBME have grown and matured over the past 40 years on substantive and methodological grounds. We believe the impact and educational utility of SBME are likely to increase in the future. More thematic programmes of research are needed. Simulation-based medical education is a complex service intervention that needs to be planned and practised with attention to organisational contexts

Effect of practice on standardised learning outcomes in simulation-based medical education

This report synthesises a subset of 31 journal articles on high-fidelity simulation-based medical education containing 32 research studies drawn from a larger qualitative review published previously. These studies were selected because they present adequate data to allow for quantitative synthesis. We hypothesised an association between hours of practice in simulation-based medical education and standardised learning outcomes measured as weighted effect sizes. Journal articles were screened using 5 exclusion and inclusion criteria. Response data were extracted and 3 judges independently coded each study. Learning outcomes were standardised using a common metric, the average weighted effect size (AWES), due to the heterogeneity of response measures in individual studies. anova was used to evaluate AWES differences due to hours of practice on a high-fidelity medical simulator cast in 5 categories. The eta squared (eta2) statistic was used to assess the association between AWES and simulator practice hours. There is a strong association (eta2=0.46) between hours of practice on high-fidelity medical simulators and standardised learning outcomes. The association approximates a dose-response relationship. Hours of high-fidelity simulator practice have a positive, functional relationship with standardised learning outcomes in medical education. More rigorous research methods and more stringent journal editorial policies are needed to advance this field of medical education research

Analysis of emergency management of acute myocardial infarction knowledge and skills decay

Introduction: Retention of cognitive knowledge and skill ability is a recognized challenge in all disciplines of health care. It is generally agreed that knowledge and skills decay over time. Many refresher-training programs for emergency medical services (EMS) personnel are offered in two-year cycles. Strategies based on the analysis of knowledge and skill decay over time can be used to improve the effectiveness and efficiency of refresher training programs. Hypothesis: In the absence of refresher training, knowledge and skills necessary for emergency management of acute myocardial infarction (AMI) will persistently decline over time. Methods: A retrospective analysis was conducted of pre- and post-test data collected over a five-year period (January 5, 1999, to March 9, 2004) from a randomized set of learners enrolled in our emergency management of AMI training program. The difference between initial post-test and next pre-test score was compared with the interval between trainings. Practicing paramedic learners were grouped into 180-day intervals for comparison and correlation. Results: The scores from 168 learners were evaluated. The average cognitive declines out of a possible 100 points for the shortest interval (180 days) and the longest interval (1,825 days) were 22.00 and 21.15, respectively. This was not statistically significant. There was no correlation in the average decline of knowledge and skill and the time interval between initial and refresher training ( r = −0.05367). Conclusion: Decline in knowledge and skill for emergency management of AMI occurs less than 180 days after initial training and reaches a nadir that remains stable over five years. Educators should evaluate and determine the knowledge and skills that are most likely to decline over time and use this information to redesign refresher-training programs. Knowledge and skill content that remains stable over time may be reduced or eliminated from refresher programs so more focus is dedicated to “decay-prone” competencies. This has tremendous implications on the development and implementation of training programs, as it has the potential to lead to more efficient programs of study

Stroke training of prehospital providers: an example of simulation-enhanced blended learning and evaluation

Since appropriate treatment of patients in the first few hours of ischemic stroke may decrease the risk of long-term disability, prehospital providers should recognize, assess, manage and communicate about stroke patients in an effective and time-efficient manner. This requires the instruction and evaluation of a wide range of competencies including clinical skills, patient investigation and management and communication skills. The authors developed and assessed the effectiveness of a simulation-enhanced stroke course that incorporates several different learning strategies to evaluate competencies in the care of acute stroke patients. The one-day, interactive, emergency stroke course features a simulation-enhanced, blended-learning approach that includes didactic lectures, tabletop exercises, and focused-examination training and small-group sessions led by paramedic instructors as standardized patients portraying five key neurological syndromes. From January to October 2000, 345 learners were assessed using multiple-choice tests as were randomly selected group of 73 learners using skills' checklists during two pre- and two post-course simulated patient encounters. Among all learners there was a significant gain in knowledge (pre: 53.9%+/-13.9 and post: 85.4%+/-8.5; p<0.001), and for the 73 learners a significant improvement in their clinical and communication skills (p<0.0001 for all). By using a simulation-enhanced, blended-learning approach, pre-hospital paraprofessionals were successfully trained and evaluated in a wide range of competences that will lead to the more improved recognition and management of acute stroke patients

Effectiveness of a cardiology review course for internal medicine residents using simulation technology and deliberate practice

Objective evaluations of residents' clinical skills reveal serious deficits. To develop, implement, and evaluate outcomes from a review course in cardiology bedside skills for internal medicine residents. We used a 1-group pretest-posttest design with historical comparisons. The study was conducted at the University of Miami School of Medicine as part of the internal medicine residency program from July 1999 to June 2000. A total of 67 2nd- and 3rd-year medicine residents received an educational intervention involving deliberate practice using simulation technology. A total of 155 4th-year medical students in one intervention and one comparison group (n = 53) served as historical comparisons. Outcome measures were a reliable computer-delivered pretest and posttest that evaluate cardiology bedside skills. Residents who received the review course and medical students who received a comparable educational intervention showed large and statistically significant pretest-to-posttest improvement in bedside skills. These 2 groups are also significantly and substantially different at posttest from a comparison group of 4th-year medical students that did not receive a specific educational intervention. Educational interventions using simulation technology that engage learners in deliberate practice of clinical skills produce large improvements in a relatively short time, with little faculty involvement