The Teacher, the Physician and the Person: How Faculty's Teaching Performance Influences Their Role Modelling

OBJECTIVE: Previous studies identified different typologies of role models (as teacher/supervisor, physician and person) and explored which of faculty's characteristics could distinguish good role models. The aim of this study was to explore how and to which extent clinical faculty's teaching performance influences residents' evaluations of faculty's different role modelling statuses, especially across different specialties. METHODS: In a prospective multicenter multispecialty study of faculty's teaching performance, we used web-based questionnaires to gather empirical data from residents. The main outcome measures were the different typologies of role modelling. The predictors were faculty's overall teaching performance and faculty's teaching performance on specific domains of teaching. The data were analyzed using multilevel regression equations. RESULTS: In total 219 (69% response rate) residents filled out 2111 questionnaires about 423 (96% response rate) faculty. Faculty's overall teaching performance influenced all role model typologies (OR: from 8.0 to 166.2). For the specific domains of teaching, overall, all three role model typologies were strongly associated with "professional attitude towards residents" (OR: 3.28 for teacher/supervisor, 2.72 for physician and 7.20 for the person role). Further, the teacher/supervisor role was strongly associated with "feedback" and "learning climate" (OR: 3.23 and 2.70). However, the associations of the specific domains of teaching with faculty's role modelling varied widely across specialties. CONCLUSION: This study suggests that faculty can substantially enhance their role modelling by improving their teaching performance. The amount of influence that the specific domains of teaching have on role modelling differs across specialties

P H D R E P O RT Evaluating clinicians' teaching performance

performance are usually a preliminary, although essential, activity in quality management and improvement activities This thesis focused on testing the validity, reliability and impact of the SETQ across specialities and settings. Most importantly, we studied (1) whether the SETQ instruments (questionnaires) can be used to evaluate clinicians' teaching performance across specialities and centres with satisfactory validity and reliability and (2) whether clinicians' teaching performance improved after receiving performance feedback. Finally, to get an indication of the predictive (criterion) validity of the SETQ evaluation scores, we studied (3) how clinicians' teaching performance scores were related to their role modelling. The system for evaluation of teaching qualities (SETQ) The SETQ was developed and preliminarily tested by Lombarts et al

Beyond limitations: Improving how we handle uncertainty in health professions education research

The researchers' assumptions invariably influence research outcomes. This is true for both qualitative and quantitative studies. Assumptions or choices regarding underlying theories, causal relations, study setting and population, sampling strategies, participant non-response, data collection, data analysis, and researchers' perceptions and interpretations of results are among factors that can induce uncertainty in research findings. Researchers tend to treat these factors as potential study limitations, but how they may impact research findings is rarely explicated and, therefore, mostly unknown. In this article, we approach uncertainty as unavoidable in research and argue that communicating about uncertainty can inform researchers, policy makers and practitioners about the validity and applicability of the study findings for their interests and contexts. We illustrate approaches to address, interpret, and explicate uncertainty in medical education research in both qualitative and quantitative paradigms. Across research paradigms, we call on researchers to consider the uncertainty in their research findings, employ appropriate methods to explore its extent and effects in their work, and communicate it explicitly in their research papers. This will help to advance our understanding of the nature and implications of the emerging knowledge in our fiel

Confirmatory Factor Analysis of the System for Evaluation of Teaching Qualities (SETQ) in Graduate Medical Training

The System for Evaluation of Teaching Qualities (SETQ) was developed as a formative system for the continuous evaluation and development of physicians' teaching performance in graduate medical training. It has been seven years since the introduction and initial exploratory psychometric analysis of the SETQ questionnaires. This study investigates the validity and reliability of the SETQ questionnaires across hospitals and medical specialties using confirmatory factor analyses (CFAs), reliability analysis, and generalizability analysis. The SETQ questionnaires were tested in a sample of 3,025 physicians and 2,848 trainees in 46 hospitals. The CFA revealed acceptable fit of the data to the previously identified five-factor model. The high internal consistency estimates suggest satisfactory reliability of the subscales. These results provide robust evidence for the validity and reliability of the SETQ questionnaires for evaluating physicians' teaching performanc

Reliable and Valid Tools for Measuring Surgeons' Teaching Performance: Residents' vs. Self Evaluation

BACKGROUND: In surgical education, there is a need for educational performance evaluation tools that yield reliable and valid data. This paper describes the development and validation of robust evaluation tools that provide surgeons with insight into their clinical teaching performance. We investigated (1) the reliability and validity of 2 tools for evaluating the teaching performance of attending surgeons in residency training programs, and (2) whether surgeons' self evaluation correlated with the residents' evaluation of those surgeons. MATERIALS AND METHODS: We surveyed 343 surgeons and 320 residents as part of a multicenter prospective cohort study of faculty teaching performance in residency training programs. The reliability and validity of the SETQ (System for Evaluation Teaching Qualities) tools were studied using standard psychometric techniques. We then estimated the correlations between residents' and surgeons' evaluations. RESULTS: The response rate was 87% among surgeons and 84% among residents, yielding 2625 residents' evaluations and 302 self evaluations. The SETQ tools yielded reliable and valid data on 5 domains of surgical teaching performance, namely, learning climate, professional attitude towards residents, communication of goals, evaluation of residents, and feedback. The correlations between surgeons' self and residents' evaluations were low, with coefficients ranging from 0.03 for evaluation of residents to 0.18 for communication of goals. CONCLUSIONS: The SETQ tools for the evaluation of surgeons' teaching performance appear to yield reliable and valid data. The lack of strong correlations between surgeons' self and residents' evaluations suggest the need for using external feedback sources in informed self evaluation of surgeons. (J Surg 69: 511-520. (C) 2012 Association of Program Director's in Surgery. Published by Elsevier Inc. All rights reserved.

Embracing the Complexity of Valid Assessments of Clinicians' Performance: A Call for In-Depth Examination of Methodological and Statistical Contexts That Affect the Measurement of Change

Assessments of clinicians' professional performance have become more entrenched in clinical practice globally. Systems and tools have been developed and implemented, and factors that impact performance in response to assessments have been studied. The validity and reliability of data yielded by assessment tools have been studied extensively. However, there are important methodological and statistical issues that can impact the assessment of performance and change that are often omitted or ignored by research and practice. In this article, the authors aim to address five of these issues and show how they can impact the validity of performance and change assessments, using empirical illustrations based on longitudinal data of clinicians' teaching performance. Specifically, the authors address the following: characteristics of a measurement scale that affect the performance data yielded by an assessment tool; different summary statistics of the same data that lead to opposing conclusions about performance and performance change; performance at the item level that does not easily translate to overall performance; how estimating performance change from two time-indexed measurements and assessing change retrospectively yield different results; and the context that affects performance and performance assessments. The authors explain how these issues affect the validity of performance assessments and offer suggestions for how to correct these issue

The Teacher, the Physician and the Person: Exploring Causal Connections between Teaching Performance and Role Model Types Using Directed Acyclic Graphs

BACKGROUND: In fledgling areas of research, evidence supporting causal assumptions is often scarce due to the small number of empirical studies conducted. In many studies it remains unclear what impact explicit and implicit causal assumptions have on the research findings; only the primary assumptions of the researchers are often presented. This is particularly true for research on the effect of faculty's teaching performance on their role modeling. Therefore, there is a need for robust frameworks and methods for transparent formal presentation of the underlying causal assumptions used in assessing the causal effects of teaching performance on role modeling. This study explores the effects of different (plausible) causal assumptions on research outcomes. METHODS: This study revisits a previously published study about the influence of faculty's teaching performance on their role modeling (as teacher-supervisor, physician and person). We drew eight directed acyclic graphs (DAGs) to visually represent different plausible causal relationships between the variables under study. These DAGs were subsequently translated into corresponding statistical models, and regression analyses were performed to estimate the associations between teaching performance and role modeling. RESULTS: The different causal models were compatible with major differences in the magnitude of the relationship between faculty's teaching performance and their role modeling. Odds ratios for the associations between teaching performance and the three role model types ranged from 31.1 to 73.6 for the teacher-supervisor role, from 3.7 to 15.5 for the physician role, and from 2.8 to 13.8 for the person role. CONCLUSIONS: Different sets of assumptions about causal relationships in role modeling research can be visually depicted using DAGs, which are then used to guide both statistical analysis and interpretation of results. Since study conclusions can be sensitive to different causal assumptions, results should be interpreted in the light of causal assumptions made in each study

A Systematic Review of the Impact of Physicians' Occupational Well-Being on the Quality of Patient Care

It is widely held that the occupational well-being of physicians may affect the quality of their patient care. Yet, there is still no comprehensive synthesis of the evidence on this connection. This systematic review studied the effect of physicians' occupational well-being on the quality of patient care. We systematically searched PubMed, Embase, and PsychINFO from inception until August 2014. Two authors independently reviewed the studies. Empirical studies that explored the association between physicians' occupational well-being and patient care quality were considered eligible. Data were systematically extracted on study design, participants, measurements, and findings. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess study quality. Ultimately, 18 studies were included. Most studies employed an observational design and were of average quality. Most studies reported positive associations of occupational well-being with patient satisfaction, patient adherence to treatment, and interpersonal aspects of patient care. Studies reported conflicting findings for occupational well-being in relation to technical aspects of patient care. One study found no association between occupational well-being and patient health outcomes. The association between physicians' occupational well-being and health care's ultimate goal-improved patient health-remains understudied. Nonetheless, research up till date indicated that physicians' occupational well-being can contribute to better patient satisfaction and interpersonal aspects of care. These insights may help in shaping the policies on physicians' well-being and quality of car

Clinical teaching performance improvement of faculty in residency training: A prospective cohort study

The purpose of this study is to investigate how aspects of a teaching performance evaluation system may affect faculty's teaching performance improvement as perceived by residents over time. Prospective multicenter cohort study conducted in The Netherlands between 1 September 2008 and 1 February 2013. Nine hundred and one residents and 1068 faculty of 65 teaching programs in 16 hospitals were invited to annually (self-) evaluate teaching performance using the validated, specialty-specific System for Evaluation of Teaching Qualities (SETQ). We used multivariable adjusted generalized estimating equations to analyze the effects of (i) residents' numerical feedback, (ii) narrative feedback, and (iii) faculty's participation in self-evaluation on residents' perception of faculty's teaching performance improvement. The average response rate over three years was 69% for faculty and 81% for residents. Higher numerical feedback scores were associated with residents rating faculty as having improved their teaching performance one year following the first measurement (regression coefficient, b: 0.077; 95% CI: 0.002-0.151; p = 0.045), but not after the second wave of receiving feedback and evaluating improvement. Receiving more suggestions for improvement was associated with improved teaching performance in subsequent years. Evaluation systems on clinical teaching performance appear helpful in enhancing teaching performance in residency training programs. High performing teachers also appear to improve in the perception of the resident