Faculty verbal evaluations reveal strategies used to promote medical student performance

Background: Preceptors rarely follow medical students’ developing clinical performance over time and across disciplines. This study analyzes preceptors’ descriptions of longitudinal integrated clerkship (LIC) students’ clinical development and their identification of strategies to guide students’ progress. Methods: We used a common evaluation framework, reporter-interpreter-manager-educator, to guide multidisciplinary LIC preceptors’ discussions of students’ progress. We conducted thematic analysis of transcripts from preceptors’ (seven longitudinal ambulatory preceptors per student) quarterly group discussions of 15 students’ performance over one year. Results: All students’ clinical development progressed, although most experienced obstacles. Lack of structure in the history and physical exam commonly obstructed progression. Preceptors used templates for data gathering, and modeling or experiences in the inpatient setting to provide time and solidify structure. To advance students’ knowledge acquisition, many preceptors identified focused learning topics with their students; to promote application of knowledge, preceptors used reasoning strategies to teach the steps involved in synthesizing clinical data. Preceptors shared accountability for helping students advance as the LIC allowed them to follow students’ response to teaching strategies. Discussion: These results depict preceptors’ perceptions of LIC students’ developmental continuum and illustrate how multidisciplinary preceptors can use a common evaluation framework to identify strategies to improve performance and follow students’ performance longitudinally

Systematic Evaluation of the Teaching Qualities of Obstetrics and Gynecology Faculty: Reliability and Validity of the SETQ Tools

BACKGROUND: The importance of effective clinical teaching for the quality of future patient care is globally understood. Due to recent changes in graduate medical education, new tools are needed to provide faculty with reliable and individualized feedback on their teaching qualities. This study validates two instruments underlying the System for Evaluation of Teaching Qualities (SETQ) aimed at measuring and improving the teaching qualities of obstetrics and gynecology faculty. METHODS AND FINDINGS: This cross-sectional multi-center questionnaire study was set in seven general teaching hospitals and two academic medical centers in the Netherlands. Seventy-seven residents and 114 faculty were invited to complete the SETQ instruments in the duration of one month from September 2008 to September 2009. To assess reliability and validity of the instruments, we used exploratory factor analysis, inter-item correlation, reliability coefficient alpha and inter-scale correlations. We also compared composite scales from factor analysis to global ratings. Finally, the number of residents' evaluations needed per faculty for reliable assessments was calculated. A total of 613 evaluations were completed by 66 residents (85.7% response rate). 99 faculty (86.8% response rate) participated in self-evaluation. Factor analysis yielded five scales with high reliability (Cronbach's alpha for residents' and faculty): learning climate (0.86 and 0.75), professional attitude (0.89 and 0.81), communication of learning goals (0.89 and 0.82), evaluation of residents (0.87 and 0.79) and feedback (0.87 and 0.86). Item-total, inter-scale and scale-global rating correlation coefficients were significant (P<0.01). Four to six residents' evaluations are needed per faculty (reliability coefficient 0.60-0.80). CONCLUSIONS: Both SETQ instruments were found reliable and valid for evaluating teaching qualities of obstetrics and gynecology faculty. Future research should examine improvement of teaching qualities when using SETQ

The Reliability of Graduate Medical Education Quality of Care Clinical Performance Measures.

BackgroundGraduate medical education (GME) program leaders struggle to incorporate quality measures in the ambulatory care setting, leading to knowledge gaps on how to provide feedback to residents and programs. While nationally collected quality of care data are available, their reliability for individual resident learning and for GME program improvement is understudied.ObjectiveTo examine the reliability of the Healthcare Effectiveness Data and Information Set (HEDIS) clinical performance measures in family medicine and internal medicine GME programs and to determine whether HEDIS measures can inform residents and their programs with their quality of care.MethodsFrom 2014 to 2017, we collected HEDIS measures from 566 residents in 8 family medicine and internal medicine programs under one sponsoring institution. Intraclass correlation was performed to establish patient sample sizes required for 0.70 and 0.80 reliability levels at the resident and program levels. Differences between the patient sample sizes required for reliable measurement and the actual patients cared for by residents were calculated.ResultsThe highest reliability levels for residents (0.88) and programs (0.98) were found for the most frequently available HEDIS measure, colorectal cancer screening. At the GME program level, 87.5% of HEDIS measures had sufficient sample sizes for reliable measurement at alpha 0.7 and 75.0% at alpha 0.8. Most resident level measurements were found to be less reliable.ConclusionsGME programs may reliably evaluate HEDIS performance pooled at the program level, but less so at the resident level due to patient volume

The Global Impact of COVID-19 on Telehealth and Care for Persons With Thoracic Cancers

Introduction: The COVID-19 pandemic has resulted in countless challenges and changes in health-systems and healthcare delivery around the world. Face-to-face consultation became the exception rather than the norm. Many people at risk of, and living with, thoracic malignancies experienced significant barriers to accessing care. Telehealth was employed by many providers to engage and monitor patients remotely, thus providing some continuity of care. The aim of this project was to assess the use of telehealth during the pandemic and the wider impact on thoracic cancer care from the perspective of healthcare professionals. Methods: An English language survey was developed by the IASLC communications committee, and administered using Qualtrics software from April 12, 2021 through May 31, 2021. It was disseminated via the IASLC and others, through multiple modalities. The 24-question survey included multiple choice, Likert scale, and free-response questions: covering two broad themes concerning the impact of the pandemic on (i) the use of telehealth and (ii) lung cancer/mesothelioma care. Some general information was also gathered such as location and primary specialty. Statistical analyses included summary statistics reported for each question by region and provider specialty, compared with Chi-Square tests. Types of Analysis and Data Reporting: Full survey results will be reported for the two study themes (i) the use of telehealth and (ii) impact on lung cancer and mesothelioma care. We will present overall results and stratify by region of the world and provider type. Statistical comparisons across groups will also be reported. Finally, free-response data will be summarized and most frequent themes identified will be reported. Results: The 141 respondents were most frequently male (63.8%), between 41-50 years old (32.6%), medical oncologists (50.7%), with majority based at academic centres (84.2%). Responses were primarily from North America (37.6%), Europe (31.2%), and Asia (14.9%). During the pandemic most used telehealth for the first time (65.2%) and billing (where appropriate) at normal rates (48.2%); the majority felt that telehealth is here to stay (48.2%). Telehealth visits were conducted by phone call (29.2%) and mixed platforms (35.7%), however ‘Video via e-medical record’ was the preferred method (42.5%). The most common barriers to adoption of telehealth were lack of resources for patients (66.1%) and regulatory limitations (56.2%), with patient interest and lack of institutional resources not rated as barriers (43.1% and 41.4%;, respectively). The top advantages for providers/patients were continuity of care and maintenance of contact with patients (88%-92% of respondents). Top disadvantages for providers were lack of human contact (72.9%), lack of patient internet access/tech knowledge (71.3%) and missing informal aspects of face-to-face visits (71.3%); these also ranked as top concerns for patients (74.8%,74%, 76.1% and 68.4%, respectively). Providers felt that telehealth was most appropriate during surveillance (94.1%) and least so for initial diagnosis (69.8%). Most felt that patients were receptive to telehealth (55.3%), however there was a worry that its use would increase healthcare disparities (29.7%). Overall, most felt that the pandemic had a negative impact on care (68%), with impacts on accessing diagnostics (i.e. biopsy), clinical trials (i.e. reduction in trials), basic/translational research (i.e. decrease in activity) as well as care (i.e. surgery). There was also a decrease in numbers accessing lung cancer screening (86.9%). Conclusion: Much will need to be done to counteract the negative impacts on care, clinical trials, and research during the COVID-19 pandemic. Although, telehealth has been widely adopted, issues remain such as healthcare access, point of use in the care pathway and telehealth platform selection