Banyard, P

Davies, MNO

Griffiths, MD

Shevlin, M

English

Nottingham Trent Institutional Repository (IRep)

PsyPAG Quarterly Newsletter – June 2003 Love me, love my lectures?  Mark Griffiths, Mark Shevlin, Philip Banyard and Mark Davies  What makes a good lecturer? Despite being one of the most asked questions in higher education, it isn’t easy to find an answer. For instance, we might value a lecturer by their ability to effect personal change and development in their students. However, this is a long-term outcome and problematic if we try to quantify it. On a practical level, one of the most popular ways of trying to answer the question of what makes a good lecturer is to ask students to rate them. However, one of the critical issues to ask when we do this, is whether we are measuring the most important variables of teaching effectiveness or whether some variables are becoming more important just because they are measurable. At the very heart of the debate is the validity of measures of teaching effectiveness gathered from student evaluations.  The practice of student evaluation of teaching (SET) in universities is ubiquitous in the UK and the US. In the UK, information from SET is considered as important evaluative information, and can be used as a guide for potential changes in course material and method of delivery. The significance of SET is also important to the Quality Assurance Agency for Higher Education (QAA) in the documentation regarding Subject Review practices (e.g., quality assessment and management issues). In the US, information from SET can be used for faculty decisions about conditions of employment such as salary and promotion. In short, SET is an integral part of higher education practices.   35PsyPAG Quarterly Newsletter – June 2003 Despite the perceived importance of SET there are many unresolved issues. First, there is little agreement on the nature and number of dimensions that represent teaching effectiveness. Measures of teaching effectiveness in the research literature have included such things as clear instructional presentation, organisation and presentation skills, management of student behaviour, intellectual excitement, respect for students, ability to challenge students, and interpersonal rapport. However, there appears to be little overlap between some of these dimensions. Furthermore, there are a number of other variables that may confound the measurement of teaching effectiveness. For example, recent research has shown that grading leniency has a strong positive relationship with ratings of teaching effectiveness. Overall, research on the effects of confounding variables on the validity of SET suggests the need for caution in the interpretation of these data. We would therefore argue that the consensus on the characteristic of effective teaching is low.  As psychologists, we also believe that if students have a positive personal and/or social view of the lecturer this may lead to more positive ratings irrespective of the actual level of teaching effectiveness. Support for this idea comes from seminal work by Solomon Asch on implicit personality theories in the 1940s. These classic studies have shown that by describing people as simply “warm” or  “cold” can produce a large effect in student judgements of lecturers. These so-called ‘halo’ and ‘horns’ effects illustrate how single attributes can be generalised to other judgements of the individual.  Because of our concerns, we recently carried out some research (published in the most recent issue of Assessment and Evaluation in Higher Education) with the hypothesis that students may respond to a  36PsyPAG Quarterly Newsletter – June 2003 central quality that then influences their evaluations of lecturers. We argued that “charisma” is such a salient trait in students' perceptions of lecturers that it affects assessment of teacher effectiveness. Based on previous research evidence and our beliefs, our study examined the relationship between charisma and teaching effectiveness and predicted that students’ perceptions of lecturers would significantly predict teaching effectiveness ratings. To do this we gave students a teaching effectiveness self-report scale, administered by members of lecturing staff. The scale was designed to measure two dimensions of teaching effectiveness. These were ‘lecturer attributes’ which measured the lecturer ability factor (e.g., “The lecturer presents material in a well organized and coherent way”, “The lecturer is able to explain difficult concepts in a clear and straightforward way” etc.) and 'module attributes' (e.g., “In this module I learned a lot”, “The references were very useful” etc.) . Students responded to these statements on a five point scale ranging from 'strongly agree' to 'strongly disagree’. An additional statement (“The lecturer has charisma”) was also included.   Using a statistical technique called structural equation modelling, our results showed that the “charisma” factor accounted for nearly 70% of the variation in ‘lecturer ability’ and 37% of the variation in the 'module attributes’. In essence, our findings indicated that student ratings don’t reflect actual teaching effectiveness, i.e., the SET ratings were significantly affected by the students' perception of the lecturer and that a ‘halo effect’ was in operation during the measurement of teaching effectiveness (because the relationships between the ‘charisma’ factor and the ‘lecturer ability’ and ‘module attributes’ were very significant).   37PsyPAG Quarterly Newsletter – June 2003 This meant that a significant proportion of the SET variation reflected a personal view of the lecturer in terms of their charisma rather than lecturing ability and module attributes.   We would be the first to argue that effective teaching is not a one-dimensional skill as it is clearly multi-dimensional. However, the issue here is about how students approach the evaluation of teaching and how they use the SET forms on the basis of a global evaluation (in this case, the charisma of the lecturer). Our study presents a challenge to the use of SET in higher education and, in particular, raises questions of fairness if such ratings are to be used in decisions relating to employment issues.   Mark Shevlin is at the School of Psychology and Communication, University of Ulster at Magee. Mark Griffiths, Philip Banyard and Mark Davies are in the Psychology Division,  Nottingham Trent University.        38

Love me, love my lectures?

Abstract

Similar works

Full text

Available Versions

Nottingham Trent Institutional Repository (IRep)