Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training?

BACKGROUND: Acquisition of competence in performing a variety of procedures is essential during Internal Medicine (IM) residency training. PURPOSES: Determine the rate of procedural complications by IM residents; determine whether there was a correlation between having 1 or more complications and institutional procedural certification status or attending ratings of resident procedural skill competence on the American Board of Internal Medicine (ABIM) monthly evaluation form (ABIM-MEF). Assess if an association exists between procedural complications and in-training examination and ABIM board certification scores. METHODS: We retrospectively reviewed all procedure log sheets, procedural certification status, ABIM-MEF procedural skills ratings, in-training exam and certifying examination (ABIM-CE) scores from the period 1990–1999 for IM residency program graduates from a training program. RESULTS: Among 69 graduates, 2,212 monthly procedure log sheets and 2,475 ABIM-MEFs were reviewed. The overall complication rate was 2.3/1,000 procedures (95% CI: 1.4–3.1/1,000 procedure). With the exception of procedural certification status as judged by institutional faculty, there was no association between our resident measurements and procedural complications. CONCLUSIONS: Our findings support the need for a resident procedural competence certification system based on direct observation. Our data support the ABIM’s action to remove resident procedural competence from the monthly ABIM-MEF ratings

Clinical Reasoning by Veterinary Students in the First-Opinion Setting: Is It Encouraged? Is It Practiced?

A mixed-methods study was performed to investigate the perceived importance and efficacy of teaching clinical reasoning (CR) skills among students and faculty in a university first-opinion veterinary practice, as this has not previously been described. Qualitative analysis of interview data, discussing objectives and factors considered important for effective learning and the understanding of CR, was performed alongside quantitative analysis of the Preceptor Thinking-Promotion Scale (PTPS) and the Learner Thinking-Behavior Scale (LTBS) (assessing the level of CR encouraged by clinicians and displayed by students) in peri-consultation discussions. Themes that emerged from analysis of the interviews regarding objectives included the desire to develop data acquisition and the need to improve data manipulation and CR. Themes associated with effective learning were a positive student-centered learning environment and feedback. Type II CR was fairly well described, but recognition of the importance of type I CR was poor among clinicians and students and, in some instances, was deemed to be inappropriate. Although many clinicians and students expressed a desire to develop student CR, there was little evidence of this actually occurring in the interactions analyzed, with low PTPS and LTBS scores achieved. There was also poor understanding of whether effective teaching of CR had occurred, demonstrated by a lack of correlation between LTBS and the interaction score for development of student CR. Further training of clinicians and students of the value of type I CR in first-opinion practice is required, as well as clinician education in how best to support the development of CR in students

A holistic review of the medical school admission process: examining correlates of academic underperformance

Background: Despite medical school admission committees’ best efforts, a handful of seemingly capable students invariably struggle during their first year of study. Yet, even as entrance criteria continue to broaden beyond cognitive qualifications, attention inevitably reverts back to such factors when seeking to understand these phenomena. Using a host of applicant, admission, and post-admission variables, the purpose of this inductive study, then, was to identify a constellation of student characteristics that, taken collectively, would be predictive of students at-risk of underperforming during the first year of medical school. In it, we hypothesize that a wider range of factors than previously recognized could conceivably play roles in understanding why students experience academic problems early in the medical educational continuum. Methods: The study sample consisted of the five most recent matriculant cohorts from a large, southeastern medical school (n=537). Independent variables reflected: 1) the personal demographics of applicants (e.g., age, gender); 2) academic criteria (e.g., undergraduate grade point averages [GPA], medical college admission test); 3) selection processes (e.g., entrance track, interview scores, committee votes); and 4) other indicators of personality and professionalism (e.g., Mayer-Salovey-Caruso Emotional Intelligence Test™ emotional intelligence scores, NEO PI-R™ personality profiles, and appearances before the Professional Code Committee [PCC]). The dependent variable, first-year underperformance, was defined as ANY action (repeat, conditionally advance, or dismiss) by the college's Student Progress and Promotions Committee (SPPC) in response to predefined academic criteria. This study protocol was approved by the local medical institutional review board (IRB). Results: Of the 537 students comprising the study sample, 61 (11.4%) met the specified criterion for academic underperformance. Significantly increased academic risks were identified among students who 1) had lower mean undergraduate science GPAs (OR=0.24, p=0.001); 2) entered medical school via an accelerated BS/MD track (OR=16.15, p=0.002); 3) were 31 years of age or older (OR=14.76, p=0.005); and 4) were non-unanimous admission committee admits (OR=0.53, p=0.042). Two dimensions of the NEO PI-R™ personality inventory, openness (+) and conscientiousness (−), were modestly but significantly correlated with academic underperformance. Only for the latter, however, were mean scores found to differ significantly between academic performers and underperformers. Finally, appearing before the college's PCC (OR=4.21, p=0.056) fell just short of statistical significance. Conclusions: Our review of various correlates across the matriculation process highlights the heterogeneity of factors underlying students’ underperformance during the first year of medical school and challenges medical educators to understand the complexity of predicting who, among admitted matriculants, may be at future academic risk

The use of contextualised standardised client simulation to develop clinical reasoning in final year veterinary students

Clinical reasoning is an important skill for veterinary students to develop before graduation. Simulation has been studied in medical education as a method for developing clinical reasoning in students, but evidence supporting it is limited. This study involved the creation of a contextualized, standardized client simulation session that aimed to improve the clinical reasoning ability and confidence of final-year veterinary students. Sixty-eight participants completed three simulated primary-care consultations, with the client played by an actor and the pet by a healthy animal. Survey data showed that all participants felt that the session improved their clinical decision-making ability. Quantitative clinical reasoning self-assessment, performed using a validated rubric, triangulated this finding, showing an improvement in students’ perception of several components of their clinical reasoning skill level from before the simulation to after it. Blinded researcher analysis of the consultation video recordings found that students showed a significant increase in ability on the history-taking and making-sense-of-data (including formation of a differential diagnosis) components of the assessment rubric. Thirty students took part in focus groups investigating their experience with the simulation. Two themes arose from thematic analysis of these data: variety of reasoning methods and “It’s a different way of thinking.” The latter highlights differences between the decision making students practice during their time in education and the decision making they will use once they are in practice. Our findings suggest that simulation can be used to develop clinical reasoning in veterinary students, and they demonstrate the need for further research in this area

Profiling strugglers in a graduate-entry medicine course at Nottingham: a retrospective case study

Background 10-15% of students struggle at some point in their medicine course. Risk factors include weaker academic qualifications, male gender, mental illness, UK ethnic minority status, and poor study skills. Recent research on an undergraduate medicine course provided a toolkit to aid early identification of students likely to struggle, who can be targeted by established support and study interventions. The present study sought to extend this work by investigating the number and characteristics of strugglers on a graduate-entry medicine (GEM) programme. Methods A retrospective study of four GEM entry cohorts (2003–6) was carried out. All students who had demonstrated unsatisfactory progress or left prematurely were included. Any information about academic, administrative, personal, or social difficulties, were extracted from their course progress files into a customised database and examined. Results 362 students were admitted to the course, and 53 (14.6%) were identified for the study, of whom 15 (4.1%) did not complete the course. Students in the study group differed from the others in having a higher proportion of 2ii first degrees, and scoring less well on GAMSAT, an aptitude test used for admission. Within the study group, it proved possible to categorise students into the same groups previously reported (struggler throughout, pre-clinical struggler, clinical struggler, health-related struggler, borderline struggler) and to identify the majority using a number of flags for early difficulties. These flags included: missed attendance, unsatisfactory attitude or behaviour, health problems, social/family problems, failure to complete immunity status checks, and attendance at academic progress committee. Conclusions Problems encountered in a graduate-entry medicine course were comparable to those reported in a corresponding undergraduate programme. A toolkit of academic and non-academic flags of difficulty can be used for early identification of many who will struggle, and could be used to target appropriate support and interventions

Adding authenticity to controlled conditions assessment: introduction of an online, open book, essay based exam

© 2018, The Author(s). In the practice of designing examinations, where the emphasis is normally on the assessment of knowledge and understanding of content covered, the authenticity of that assessment model related to skills required for employability, as well as our growing access to instant information, is often missed. Some traditional methods of this form of assessment, particularly the paper based essay exam, have expectations of writing quality and structure as well as of knowledge and understanding, but without providing the tools with which to appropriately meet those expectations. To address this, a computer based, unseen, essay format exam was introduced to a final year module for applied science students, where access was permitted to specific online journal resources. Students wrote their submissions under controlled conditions in Microsoft Word and submitted it through the Virtual Learning Environment (VLE), Blackboard for grading. In addition to an improvement in average mark across the cohort compared to the previous year of paper based essay examinations on the same module, student feedback highlighted an improvement in their ability to organise and arrange their knowledge, and the usefulness of using research material to better evidence their own knowledge. Feedback, being completed online and returned to the students also provided greater understanding of grades achieved and of how to improve for future examinations, a unusual process for exams where graded papers are not normally returned. Though upskilling of staff is likely required for more widespread use, and repeat cohorts will no doubt demonstrate further pros and cons of this form of assessment, the model presented here shows promise for assessment of student communication of depth and breadth of knowledge, and of demonstrating skills more relevant to future employment

Situational awareness within objective structured clinical examination stations in undergraduate medical training - a literature search

Background: Medical students may not be able to identify the essential elements of situational awareness (SA) necessary for clinical reasoning. Recent studies suggest that students have little insight into cognitive processing and SA in clinical scenarios. Objective Structured Clinical Examinations (OSCEs) could be used to assess certain elements of situational awareness. The purpose of this paper is to review the literature with a view to identifying whether levels of SA based on Endsley's model can be assessed utilising OSCEs during undergraduate medical training. Methods: A systematic search was performed pertaining to SA and OSCEs, to identify studies published between January 1975 (first paper describing an OSCE) and February 2017, in peer reviewed international journals published in English. PUBMED, EMBASE, PsycINFO Ovid and SCOPUS were searched for papers that described the assessment of SA using OSCEs among undergraduate medical students. Key search terms included "objective structured clinical examination", "objective structured clinical assessment" or "OSCE" and "non-technical skills", "sense-making", "clinical reasoning", "perception", "comprehension", "projection", "situation awareness", "situational awareness" and "situation assessment". Boolean operators (AND, OR) were used as conjunctions to narrow the search strategy, resulting in the limitation of papers relevant to the research interest. Areas of interest were elements of SA that can be assessed by these examinations. Results: The initial search of the literature retrieved 1127 publications. Upon removal of duplicates and papers relating to nursing, paramedical disciplines, pharmacy and veterinary education by title, abstract or full text, 11 articles were eligible for inclusion as related to the assessment of elements of SA in undergraduate medical students. Discussion: Review of the literature suggests that whole-task OSCEs enable the evaluation of SA associated with clinical reasoning skills. If they address the levels of SA, these OSCEs can provide supportive feedback and strengthen educational measures associated with higher diagnostic accuracy and reasoning abilities. Conclusion: Based on the findings, the early exposure of medical students to SA is recommended, utilising OSCEs to evaluate and facilitate SA in dynamic environment