Using educational prescriptions to teach medical students evidence-based medicine

<p><b>Purpose</b>: To evaluate feasibility and impact of evidence-based medicine (EBM) educational prescriptions (EPs) in medical student clerkships.</p> <p><b>Methods</b>: Students answered clinical questions during clerkships using EPs, which guide learners through the “four As” of EBM. Epidemiology fellows graded EPs using a rubric. Feasibility was assessed using descriptive statistics and student and fellow end-of-study questionnaires, which also measured impact. In addition, for each EP, students reported patient impact. Impact on EBM skills was assessed by change in EP scores over time and scores on an EBM objective structured clinical exam (OSCE) that were compared to controls from the prior year.</p> <p><b>Results</b>: 117 students completed 402 EPs evaluated by 24 fellows. Average score was 7.34/9.00 (SD 1.58). 69 students (59%) and 21 fellows (88%) completed questionnaires. Most students thought EPs improved “Acquiring” and “Appraising”. Almost half thought EPs improved “Asking” and “Applying”. Fellows did not value grading EPs. For 18% of EPs, students reported a “change” or “potential change” in treatment. 56% “confirmed” treatment. EP scores increased by 1.27 (95% CI: 0.81–1.72). There were no differences in OSCE scores between cohorts.</p> <p><b>Conclusions</b>: Integrating EPs into clerkships is feasible and has impact, yet OSCEs were unchanged, and research fellows had limitations as evaluators.</p

How Much Can Students Gain in Data Analysis and Critical Thinking Skills in One Semester?

Background/Question/Methods: The effective preservation and sustainable use of ecosystems is a complex endeavor that requires proficiency in skills of critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, there is concern that US undergraduate science students do not currently develop these fundamental process skills they will need as professionals. In this study, we investigate how we can best ‘operationalize’ the teaching of process skills and how we can assess their development in undergraduate students. We are implementing a multi-year, multi-institutional research project to: (1) develop a set of instructional materials and assessment tools for critical thinking, oral communication, and data analysis; and (2) pilot these materials in a diversity of classroom settings under two instructional modalities: individual student reflection versus intensive classroom discussion of the skill. Results/Conclusions: Twenty-four conservation biologists have collaborated during the last year to create and validate instructional materials for process skills development, led by the Network of Conservation Educators and Practitioners (ncep.amnh.org). The instructional set for each skill consisted of pre/post student self-assessments, two exercises with rubrics for evaluation of student performance, and pre/post exercise content assessments. In fall 2011, nine professors piloted these materials in biology, ecology, and conservation biology courses. We present preliminary results from a subset of their students, from instructional units on data analysis using an intensive classroom discussion (DA; N=22) and critical thinking using individual student reflection (CT; N=20). For DA, we find significant increases in student self-confidence on data representation and interpretation (P0.05). For both skills, we find gains in content knowledge after the application of exercises (DA: g=0.22±0.082; CT: g=0.4±0.11). Observed gains in the skills vary depending on the aspect analyzed. For DA, students experienced significant gains in data representation and interpretation (V=17,

Cultivating Skills for 21st Century Professionals: Development and Assessment of Process Skills in Ecology and Conservation Biology Students

Background/Question/Methods: Understanding and conserving the biosphere increasingly requires proficiency in skills including critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, a real concern exists that US undergraduate science students do not currently develop these important process skills needed as professionals. First, our work asks “how can we best ‘operationalize’ teaching of process skills?” Second, we must also discover how to evaluate development of specific process skills in our students. Such assessment allows us to determine if we have succeeded in teaching these skills with a gain or loss in content understanding. Here, we present the design and preliminary results of a recently launched experimental study aimed at these two tasks. The study brings together faculty from diverse institutions and professional conservation biologists to create and validate a set of instructional materials for process skills development. Following development, selected faculty will pilot teaching and assessment materials in diverse classroom settings. Project participants, led by investigators from the American Museum of Natural History’s Center for Biodiversity and Conservation, designed a multi-year research experiment in two stages: (1) development of instructional materials and associated assessment tools for three skills (i.e., critical thinking, oral communication, and data analysis), and (2) application of these materials in the classroom under two different instruction modalities, individual reflection versus intensive classroom discussion of the skill. These two modalities investigate student development of the targeted process skill and how intensity of a teaching intervention influences student success. Results/Conclusions: For each selected skill, faculty participants and consultants developed two exercises that focused on ecology and conservation biology topics, either new or based on the modules produced by the Network of Conservation Educators and Practitioners (NCEP; http://ncep.amnh.org). Participants also created the following assessment tools: student self-assessment questionnaires focusing on their confidence for each skill, content gains assessments, and rubrics for assessing skill performance. These will be applied pre and post application of the instructional materials. Our plans include validation and finalization of materials over summer 2011 and pilots in the classrooms occurring in at least 14 US institutions (including Puerto Rico) beginning in fall 2011. While the study focuses on these skills in the context of ecology and conservation biology, our approach and results should be of direct use for other related, integrative fields such as natural resource management, sustainable development studies, and public health. We welcome faculty input and interest in the project