A Good Way to Disseminate Your University of Michigan Academic Products to the General Public Through Deep Blue and Creative Commons

Deep Blue is a University of Michigan library system that permanantly stores electonic versions of faculty-generated content. Creative Commons is an addendum to copyright that gives people permission to disseminate and use your work. Together, these make a public publishing platform that's pretty easy to set up for academic posters, talk slides, audio recordings, published or unpublished manuscripts, data sets, what have you.http://deepblue.lib.umich.edu/bitstream/2027.42/91292/1/stansfield-2012-abstract.txthttp://deepblue.lib.umich.edu/bitstream/2027.42/91292/3/howtodeepblueit.pd

Conditional reliability of admissions interview ratings: extreme ratings are the most informative

Admissions interviews are unreliable and have poor predictive validity, yet are the sole measures of non-cognitive skills used by most medical school admissions departments. The low reliability may be due in part to variation in conditional reliability across the rating scale. Objectives  To describe an empirically derived estimate of conditional reliability and use it to improve the predictive validity of interview ratings. Methods  A set of medical school interview ratings was compared to a Monte Carlo simulated set to estimate conditional reliability controlling for range restriction, response scale bias and other artefacts. This estimate was used as a weighting function to improve the predictive validity of a second set of interview ratings for predicting non-cognitive measures (USMLE Step II residuals from Step I scores). Results  Compared with the simulated set, both observed sets showed more reliability at low and high rating levels than at moderate levels. Raw interview scores did not predict USMLE Step II scores after controlling for Step I performance (additional r 2  = 0.001, not significant). Weighting interview ratings by estimated conditional reliability improved predictive validity (additional r 2  = 0.121, P  < 0.01). Conclusions  Conditional reliability is important for understanding the psychometric properties of subjective rating scales. Weighting these measures during the admissions process would improve admissions decisions.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/71581/1/j.1365-2929.2006.02634.x.pd

Assessing the Learning Environment at the University of Michigan Medical School Through a National Collaboration

The Learning Environment Study involves 28 medical schools belonging to the Innovative Strategies for Transforming the Education of Physicians (ISTEP): an initiative founded by the American Medical Association in 2006. ISTEP is a unique medical education research collaborative that brings together individuals and institutions across the continuum of student/physician learning with a mission to foster evidence-based changes in physician education that will improve patient care. In early 2010, ISTEP developed a protocol to examine the undergraduate medical education environment: a prospective, repeated measures, longitudinal research design, employing a diverse set of established measures. The class of 2014 became the first cohort enrolled in this study, with 11 schools participating. A second cohort from the class of 2015 became the second cohort with 25 schools participating. The results reported here summarize the University of Michigan Medical School data for student empathy, patient-provider orientation, ways of coping, tolerance for ambiguity, and their perceptions of the learning environment at UMMS.http://deepblue.lib.umich.edu/bitstream/2027.42/91290/1/poster1.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/91290/3/MEDI01poster.pd

Medical Students' Problem-Solving Skills Predict How They Experience Medical School

Ob jective The learning environment impacts medical students’ motivation, attitudes, academic performance, and professionalism. Aspects of the learning environment— faculty and administrative treatment of students, student social experiences, consequences for intellectual honesty and ethical integrety—constitute a “hid- den curriculum” which every medical school must strive to improve. As part of the American Medical Association’s Innovative Strategies for Transforming the Education of Physicians (ISTEP) initiative, the Learning Environment Study (LES), we sought psychological predictors of student subjective experience of the learning environment. We predicted that a measure of students’ tendency to react to difficult situations immediately upon matriculation would predict how they rated various aspects of their medical school experience at the end of their second year. Methods 155 undergraduate medical students from the classes of 2014 and 2015 com- pleted a subset of the Ways of Coping Scale (WCS: 22 items comprising 8 validated subscores, Folkman & Lazarus, 1986) at the time of matriculation and the Medical Student Learning Environment Scale (MSLES: 17 items an- alyzed individually, Rosenbaum, et al, 2007) at the end of their second year. We performed multiple regression Ratings on each MSLES item by the 8 WCS subscores. Results Social aspects of medical school were rated more positively by students with higher Planful Problem-Solving and Positive Reappraisal scores: these students make conscious efforts to actively, productively address their difficult situations. In contrast, students with high Escape Avoidance scores gave lower ratings for “Students gather together for informal activities.” Students who were more emotionally controlled (high Self-Controlling scores) found the school administration less likely to take meaningful action to support students and that students were reluctant to share their troubles with each other. Student who tend to seek Social Support when dealing with problems had difficulties finding time for interests outside of medicine and reported intense competition for grades. Conclusions Students’ experience of medical school is to some extent a function of their general approach to problem-solving. To improve the learning environment, schools should be sensitive to students’ differences in this regard. Encouraging students to use planful problem-solving and positive reappraisal may have pos- itive benefits. This analysis represents one only school and an analysis of the larger multi-institutional ISTEP LES dataset may challenge the consistency of these results across schools.http://deepblue.lib.umich.edu/bitstream/2027.42/97017/1/Stansfield2013WaysofCoping-abstract.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/97017/3/CGEA2013-Stansfield.pd

Progress Testing of Basic Science Application During a Clinical Competency-Based Curriculum Pilot

Background: In a clinical comptentency-based curriculum, assessment of students' basic science knowledge is important. During a feasibility pilot of such a curriculum, six students acquired basic science knowledge about three clinical domains (renal failure, trauma, hyperglycemia) from clinical encounters with real patients as well as traditional learning resources. Summary of Work: Multiple choice items used for progress tests at Peninsula Medical School were assembled into three 50 item tests, administered online at 2-week intervals. Items used clinical scenarios to test knowledge in the 3 domains plus a control domain (infectious disease). Complete item and answers were provided after each test as formative feedback. Summary of Results: Tests were difficult (mean topic scores from 32\% to 80\%). Repeated items were easier (80\%) than novel items (58\%). Scores on items that specifically addressed learning objectives improved, though not statistically significantly. Conclusions: Brief progress tests are feasible assessments of student learning, though the reliability is a concern in this pilot. Take home messages: Short progress tests may not be the most sensitive way to evaluate basic science knowledge in a clinical competence curriculum. Longer or more focused tests given at longer intervals may be more sensitive to knowledge application abilities acquired during training.http://deepblue.lib.umich.edu/bitstream/2027.42/71389/3/article.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/71389/1/slides.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/71389/7/stansfield01.pd

Assessment of musculoskeletal examination skills of 4th year medical students using a novel OSCE

Objective: Despite the high prevalence of musculoskeletal complaints presenting to physicians in the United States, there are very few opportunities for University of Michigan clinical medical students to receive formative or summative assessment of their ability to evaluate patients with these complaints. The purpose of this study was to assess 4th year students’ ability to examine and diagnose several common musculoskeletal disorders using a novel objective structured clinical examination (OSCE). Methods: A multidisciplinary team of musculoskeletal specialists developed the content and structure of three OSCE stations focusing on examination of the shoulder, back and knee. For each station, volunteer M4 students were provided a clinical vignette with three possible diagnoses to consider, and were instructed to anticipate physical examination maneuvers or findings that would discriminate between the three diagnoses. Then they would examine a professional patient simulating findings associated with one of the diagnoses and choose their favored diagnosis. Their encounter was directly observed by a faculty member who scored their performance on selected physical examination maneuvers based on a checklist (0 = not done, 1 = partially done, 2 = fully done). Each encounter was recorded to allow for later review by another faculty. Immediate feedback was provided to students at the end of the OSCE, making this a formative as well as summative assessment experience. Faculty received verbal and written instruction on how to score students. IRB exemption was obtained for this study. Results: 44 M4 students participated in the OSCE during the spring of 2012. General performance of M4 students in examining regional musculoskeletal complaints will be reported. Performance of individuals will be correlated with: anticipation of discriminatory features prior to examining the patients; self-assessment on ability to perform the relevant exam and anticipated need to do so in their future career; previous musculoskeletal elective exposure; future career choice; and performance on the M4 Comprehensive Clinical Assessment “Back pain” and “Abdominal pain” stations. Conclusions: Initial validity evidence for a multistation musculoskeletal OSCE will be presented, as will the performance of a sampling of the 2012 graduating UM medical student class. This data will be used as part of ongoing evaluation of the longitudinal musculoskeletal curriculum at the University of Michigan medical school.http://deepblue.lib.umich.edu/bitstream/2027.42/91291/1/MedEdDay2012-poster-monradetal.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/91291/3/MEDC22poster.pd

Faculty Evaluation of Student Portfolio Presentations of a Seven-Week Clinical Competency-Based Curriculum Pilot

Background: Student portfolios intend to capture qualitative aspects of students' learning experiences, and foster personal responsibility for learning. However, obtaining standardized assessments of portfolios is challenging. This challenge must be met for a true competency-based curriculum in which students have individualized learning paths and educational goals. Summary of Work: During a six-week pilot of a flexible, competency-based curriculum, 5 students summarized their learning experiences for 7 faculty raters who rated each using a novel 5-item instrument. Presentations were 20 minutes long followed by a 10 minute question period. Follow up ratings of videotaped presentations were used to resolve rater disagreements and improve the rating form. Summary of Results: There was low inter-rater reliability of the rating instrument (item intra-class correlations (ICC) ranged from .00 to .91). Follow-up ratings found agreement easier to reach with better-defined item anchors. Conclusions: The difficulties underlying summative assessment of an inherently qualitative experience are likely surmountable. Allowing students to defend their academic progress to a faculty panel in person is enjoyable and worthwhile. Further refinement of a rating instrument will likely overcome interrater reliability issues. Take home messages: Standardized global performance assessment of individualized learning paths is feasible using faculty ratings of semi-structured student presentations.http://deepblue.lib.umich.edu/bitstream/2027.42/76026/1/stansfield02.dochttp://deepblue.lib.umich.edu/bitstream/2027.42/76026/4/stansfield02.pd

Development of the Resident Wellness Scale for Measuring Resident Wellness

Purpose: Graduate medical education programs have a responsibility to monitor resident wellness. Residents are at risk of burnout, depression, and suicide. Burnout and depression are associated with poor patient care. Many existing tools measure burnout, depression, and general human well-being, but resident wellness is a distinct construct. We aimed to develop an instrument to measure resident wellness directly. Methods: An expert panel from two purposefully different graduate medical education institutions generated a behavior- and experience-based model of resident wellness. The panel and resident leaders from both institutions generated 92 items, which were tested alongside anchor scales measuring burnout, depression, personality, optimism, life satisfaction, and social desirability in a convenience sample of 62 residents. Ten items were selected using a combination of factor analysis, a genetic algorithm, and purposeful selection. The 10-item scale was distributed to 5 institutions at which 376 residents completed it anonymously. Exploratory factor analysis was used to examine the factor structure of the scale. Results: The model of resident wellness aligned with an accepted framework of well-being in the literature. The 10-item Resident Wellness Scale broadly covered the model and correlated meaningfully with anchor scales. The factor structure of the scale suggested sensitivity to meaningful work, life security, institutional support, and social support. Conclusions: This novel Resident Wellness Scale is designed to track residents’ wellness longitudinally. It is sensitive to aspects of resident wellness that have been shown to reduce burnout and depression and appears to be a psychometrically strong measure of resident wellness

The Leadership Inventory for Medical Education (LIME): A Novel Assessment of Medical Students’ Leadership Skills

Purpose: We sought to develop a novel measurement instrument for leadership knowledge, skills and behaviors of medical students as part of the program evaluation for a curriculum redesign. Method: The Leadership subgroup of our curriculum redesign process generated a definition of leadership consisting of four domains: 1) leading teams, 2) systems based practice, 3) influence and communication, and 4) problem solving. The definition of each domain was used to generate a 12-item instrument (the Leadership Inventory for Medical Education, or LIME) with items rated on a 4-point frequency scale. 315 medical students from two cohorts at a large midwestern university medical school completed the instrument at matriculation and at the end of the M1 and M2 year with an abbreviated version of the Ways of Coping Scale. The 4 domain structure of the LIME was tested using Confirmatory Factor Analysis and correlations of LIME subscores with 8 Ways of Coping were computed as evidence of construct validity. Results: The LIME showed acceptable unidimensionality with Cronbach’s alpha = .79 and a four-factor structure closely matching the four target domains. Subscores were derived from the observed factor structure: Analysis, Culture, Policy, and Communication. LIME scores increased slightly but non-significantly from matriculation to the end of M1 except for Communication scores which rose significantly (t(141) = 2.13, p < 0.05). LIME scores correlated positively with proactive coping styles (Seeking Social Support and Planful Problem-Solving) and negatively with emotional, passive styles (Distancing). Conclusions: Given the growing importance of teamwork and the rapidly shifting landscape of healthcare, medical students will need strong leadership skills to be clinically effective in their careers. Despite this need, little work has been done to research the effectiveness of curricular strategies for developing leadership. The LIME may be a useful tool for measuring and tracking students’ leadership skills during their professional development.http://deepblue.lib.umich.edu/bitstream/2027.42/115886/1/MEDC24LeadershipScale- 3.24.15.docxhttp://deepblue.lib.umich.edu/bitstream/2027.42/115886/3/AAMC2015-LIME-poster.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/115886/4/Leadership.pdfDescription of MEDC24LeadershipScale- 3.24.15.docx : AbstractDescription of AAMC2015-LIME-poster.pdf : Poster at AAMC/RIME 2015Description of Leadership.pdf : The Leadership Inventory for Medical Education (LIME

An Objective Structured Clinical Examination Case for Opioid Management: Standardized Patient Ratings of Communication Skills as a Predictor of Systems-Based Practice Scores

The Wayne State University Office of Graduate Medical Education (WSUGME) uses an objective structured clinical examination (OSCE) to assess its programs’ contribution to enhancing residents’ communication skills. In response to revisions in Michigan’s opioid-prescribing mandates in 2017, WSUGME developed a pain management case in collaboration with faculty and the Wayne State University School of Medicine to educate residents about these mandates while gauging their skills in Systems-Based Practice (SBP), an Accreditation Council for Graduate Medical Education Core Competency. This study examined whether resident OSCE performance predicted year-end milestones scores in SBP1 (coordinates patient care within various health care delivery settings), SBP2 (works in interdisciplinary teams to enhance patient safety and improve patient care quality), and SBP3 (practices and advocates for cost-effective, responsible care). Participants included two cohorts of first- (PRG-1) and second-year (PRG-2) residents in 6 programs: one cohort from academic year 2018-2019 (n = 33), the other from 2019-2020 (n = 37). Before the OSCE, WSUGME emailed residents the new state prescription requirements. During the simulated encounter, standardized patients rated residents on a validated communication instrument, and WSUGME conducted a linear regression of patient ratings on resident SBP milestone scores. The ratings of communication skills of PRG-1 residents did not predict any of the year-end SBP milestones. However, ratings of communication skills of PRG-2 residents predicted SBP1 and SBP2, though not SBP3, milestones. The OSCE opioid case proved to be a valid measure of PRG-2 residents’ competence gained across the first year but was less meaningful when applied to PRG-1 residents