Leniency and halo effects in marking undergraduate short research projects

BACKGROUND: Supervisors are often involved in the assessment of projects they have supervised themselves. Previous research suggests that detailed marking sheets may alleviate leniency and halo effects. We set out to determine if, despite using such a marking schedule, leniency and halo effects were evident in the supervisors' marking of undergraduate short research projects (special study modules (SSM)). METHODS: Review of grades awarded by supervisors, second markers and control markers to the written reports of 4(th )year medical students who had participated in an SSM during two full academic years (n = 399). Paired t-tests were used to compare mean marks, Pearson correlation to look at agreement between marks and multiple linear regression to test the prediction of one mark from several others adjusted for one another. RESULTS: There was a highly significant difference of approximately half a grade between supervisors and second markers with supervisors marking higher. (t = 3.12, p < 0.01, difference in grade score = 0.42, 95% CI for mean difference 0.18–0.80). There was a high correlation between the two marks awarded for performance of the project and the written report by the supervisor (r = 0.75), but a low-modest correlation between supervisor and second marker (r = 0.28). Linear regression analysis of the influence of the supervisors' mark for performance on their mark for the report gave a non-significant result. This suggests a leniency effect but no halo effect. CONCLUSIONS: This study shows that with the use of structured marking sheet for assessment of undergraduate medical students, supervisors marks are not associated with a halo effect, but leniency does occur. As supervisor assessment is becoming more common in both under graduate and postgraduate teaching new ways to improve objectivity in marking and to address the leniency of supervisors should be sought

Audit and feedback and clinical practice guideline adherence: Making feedback actionable

BACKGROUND: As a strategy for improving clinical practice guideline (CPG) adherence, audit and feedback (A&F) has been found to be variably effective, yet A&F research has not investigated the impact of feedback characteristics on its effectiveness. This paper explores how high performing facilities (HPF) and low performing facilities (LPF) differ in the way they use clinical audit data for feedback purposes. METHOD: Descriptive, qualitative, cross-sectional study of a purposeful sample of six Veterans Affairs Medical Centers (VAMCs) with high and low adherence to six CPGs, as measured by external chart review audits. One-hundred and two employees involved with outpatient CPG implementation across the six facilities participated in one-hour semi-structured interviews where they discussed strategies, facilitators and barriers to implementing CPGs. Interviews were analyzed using techniques from the grounded theory method. RESULTS: High performers provided timely, individualized, non-punitive feedback to providers, whereas low performers were more variable in their timeliness and non-punitiveness and relied on more standardized, facility-level reports. The concept of actionable feedback emerged as the core category from the data, around which timeliness, individualization, non-punitiveness, and customizability can be hierarchically ordered. CONCLUSION: Facilities with a successful record of guideline adherence tend to deliver more timely, individualized and non-punitive feedback to providers about their adherence than facilities with a poor record of guideline adherence. Consistent with findings from organizational research, feedback intervention characteristics may influence the feedback's effectiveness at changing desired behaviors

Psychological and physical well-being during unemployment: A meta-analytic study

JOURNAL OF APPLIED PSYCHOLOGY90153-76JAPG

Is the Annual Confidential Report system effective? A study of the government appraisal system in Gujarat, India.

BACKGROUND Effective performance appraisal systems can not only motivate employees to improve performance but also be important for the performance of organizations. However, the appraisal systems in civil services called the Annual Confidential Report (ACR) systems can be ineffective and do not contribute to employees' learning and development. With this background, the current study aimed at understanding the ACR system and assessing its effectiveness. The research aims to contribute in filling the knowledge gap in the existing literature on the need as to why the ACR system in civil services is an important human resource management (HRM) function. METHODS The analysis is based on policy review to understand the extant appraisal-related rules and policies. Nineteen in-depth interviews with medical officers (MOs) working with the government health department of Gujarat, India, were conducted. The main objective of the research was to assess the effectiveness of the actual appraisal system called or referred to as the ACR as perceived by MOs. Thematic framework approach was used to analyze qualitative data using NVIVO 9. Themes were built around five features of an effective appraisal system, i.e., purpose, source, feedback quality, link of the ACR system with other human resource functions, and administrative effectiveness. RESULTS The five features of the effective appraisal system studied in the current research (purpose, source, feedback quality, link of ACR system with other HRM functions, and administrative effectiveness) indicate that the overall appraisal system is ineffective. The overall appraisal system was perceived to be subjective and one directional in character by the study respondents. Furthermore, respondents perceived the appraisal system to be a ritual and where MOs hardly got to know about their performance, especially good performance. Hence, the feedback loop, an important feature for an effective appraisal system, was absent. The overall ACR system functions in isolation with no link to other HRM functions such as training and counselling, and a weak link with salary administration and promotion. CONCLUSIONS Addressing the five features or domains of an effective appraisal system can lead to improved perceived fairness MOs have on the current appraisal system which may further influence the satisfaction and motivation positively. Improved motivation and satisfaction with the appraisal system can influence two important human resource for health-related outcomes, i.e., performance and retention