Cogdell, Barbara

Finlay, Chris

McVey, Mary

English

Enlighten

          Cogdell, Barbara, Finlay, Chris, and McVey, Mary (2011) Peer-assessment of group work in a large class: development of a staff and student friendly system. In: Orsmond, Paul (ed.) Self- and Peer-Assessment: Guidance on Practice in the Biosciences. Series: Teaching Bioscience: Enhancing Learning Series . The Higher Education Academy, UK Centre for Bioscience,, Leeds, pp. 49-51. ISBN 9780956928832   Copyright © 2011 The Higher Education Academy   A copy can be downloaded for personal non-commercial research or study, without prior permission or charge  Content must not be changed in any way or reproduced in any format or medium without the formal permission of the copyright holder(s)   When referring to this work, full bibliographic details must be given    http://eprints.gla.ac.uk/90943/      Deposited on: 07 March 2014                Enlighten – Research publications by members of the University of Glasgow http://eprints.gla.ac.uk Background and Rationale: First year biology at the University of Glasgow is divided into two sequential courses, which both have a group project assignment worth 10% of the summative assessment. We decided to use peer assessment for these group assignments, not only to cope with the fact that it would be impossible for the staff to directly monitor the activity of every group, but also to enhance the student learning experience.  There are between 600 and 700 students taking each course. They are divided into a maximum of 16 laboratory classes with about 48 students in each. Each laboratory class is then further divided into six groups of eight students, giving a possible maximum of 96 groups. The students are timetabled for specific laboratory times, and have been allocated specific laboratory seats and so become familiar with one another from the beginning of the academic session. The first assignment is based on ‘Darwin’s Dilemma’ and requires students (in their groups of up to eight) to prepare a 5 minute presentation arguing the case for the elimination of an invasive organism from a chosen geographical area. The presenting group are then questioned by an ‘expert panel’ who will test their knowledge of the particular organism. This expert panel is made up of fellow students from another group. Consequently every student group will act as a presenting group and as an expert panel for another group. A similar format is repeated for the second group assignment, which is to select an important biological molecule and construct a poster showing its structure; how and where it is produced; and its biological role. The task is completed by a 5 minute presentation defending the poster’s content with questions from an expert panel as before. The groups meet both in scheduled laboratory classes and in their own study time to research their chosen topics and complete their tasks. Though the scheduled laboratory classes are managed by members of staff, the staff cannot easily monitor the process when the students meet outside of their timetabled classes. Several tools are used both to encourage the students to monitor themselves and to allow staff to ‘follow’ the progress of the group work. Each group is required to elect a group leader and deputy. These individuals’ details are recorded for each group. The leaders are asked to note any relevant factors (e.g. people who seldom attend meetings; people who were helpful in researching the topic but not present at the presentation, etc.) as they may be consulted in the event of erratic peer marking. In order to assist with the communication within a group staff set up a specific Moodle forum for each group. Moodle is the VLE used at the University of Glasgow. These specific forums allow group members to discuss their work with each other without allowing students from other groups to see their ‘posts’. However staff can see all the ‘posts’ and can use these as a way of monitoring group progression. A series of deadlines for the group task are posted on Moodle. Each group is required to ‘post’ the division of responsibilities of its members. All students are made aware of the peer-marking aspect of the assignments and each group is asked to ‘post’ their agreed marking criteria onto their forum. This can then be referred to by staff if required. The marking criteria will vary across groups and usually include points such as attendance at group meetings. There is a staff marking element to the groups’ presentations and expert panel questioning. Two members of staff are present for each presentation and agree a score for the group’s presented information and their ability to answer to questions from the expert panel.  The criteria that staff use for these scores, e.g. audibility, links to research, time keeping, etc., are made available to students several weeks before the presentations themselves. This staff mark is only part of the scoring used to generate a final, student specific grade. As a result of previous experience with group work it was also felt to be important to introduce a method of distinguishing individual contributions, i.e. to introduce peer-assessment. In an original group work activity there had been continuous complaints that the staff did not assess the group work and that the group members did not like carrying non-contributors. In the last couple of years of the original version of group work a high proportion of the students failed to contribute to the group activity. Development Through Experience: For the first introduction of peer-assessment each group was awarded a staff mark out of 100% for their work. This mark was multiplied by the number of members in the group to give the total number of marks that would be distributed across the group. The groups were then made aware of this total score and asked to divide the marks between themselves. For example, if a group has eight members and receives a staff score of 60% the total number of marks to be distributed across the group members is 480. The students allocated their peer marks together as a group in a scheduled laboratory session. Each group was given a single form with the full names and student ID numbers of each of their fellow members and a space to indicate their marks. These completed sheets were collected by the staff member leading the laboratory. It was noted that many groups agreed to share the marks equally amongst themselves. This scheme was fairly simple to run as there was an agreed sheet of marks per group and staff subsequently input these agreed marks into the assessment spreadsheet. Checks were made to ensure that the students had made correct calculations. Any queries could be sorted by consultation with the group leader and any students awarded zero by their group were investigated by staff for any extenuating circumstances.  However the students did not like this scheme. Allocating marks in a group setting made it difficult and awkward to award low marks. Consequently non-contributors would get the same marks as everyone else and group members often felt resentful. Some groups will always be dysfunctional and as the mark contributes to their final course grade, it is unfair to tell the students to sort this out by themselves. A lot of additional staff work was generated trying to address issues like this. Next Step – Make the marking confidential and automate the marks calculation: The department’s systems analyst devised forms using Microsoft Word with mail merge, which could be fed into an Intelligent Character Recognition (ICR) machine. This machine reads paper forms with text entries electronically and the scores may then be transferred directly into an Excel spreadsheet. All subsequent calculations can be completed on the spreadsheet. Using these forms made it feasible for each student to submit their marks confidentially. As before the group work was awarded a staff score out of 100%. Again this score was multiplied by the number of students to give the total number of group marks for distribution. Students submitted their individual scoring forms, marking each of their fellow group members out of 10. These forms were read electronically and an average peer mark (out of 10) was calculated for each student. All the average marks for the group were added together to give a sum of peer marks for the whole group. This was then used to calculate the proportion of the total peer marks that should be awarded to each individual. Although it is possible to get over 100% with this formula final marks were capped at 100%. A major advantage of this system is that it takes into account whether the students are harsh or lenient markers. Obviously the system will not work if students fail to return their forms and so students were informed that failure to submit their scores would result in no marks being awarded for their work. The students were much happier with this system and appreciated being able to reward hard work and penalise freeloaders. There were favourable reports from both the end-of-course questionnaires and staff-student liaison meetings. Some students with low grades did complain, but because group leaders had been instructed to note attendance at group meetings it was relatively easy to point out to them that they had contributed very little and they usually agreed without further complaint. This second iteration solved the student identified issues, but still involved a major administrative task for staff. The ICR system is very efficient at reading the student forms, but relies on students using legible script and filling all boxes correctly. Each time the forms were illegible or filled in wrongly they had to be checked by the operator. With the large student numbers involved this became very onerous. Latest Development – Move the submission form and admin online: In an attempt to reduce staff time the system has developed to include online administration and handling of student scoring. Now the students are required to enter their marks for their colleagues using a web-based form. Each student is e-mailed a unique URL code, which is generated from their student ID number. This URL allows students to access their own specific webpage with a webform with a pre-generated list of their other group members and spaces to enter their marks. The webform asks each student to distribute 100 marks across the other members of their group. The webform has built-in validation and so cannot be submitted with any blank fields or if the score allocation does not add up to 100. In addition, each student is also asked to award themselves a mark out of 10 and include a paragraph explaining their self awarded score. This score is not used in the final allocation of marks but can be referred to, if a student feels that their final grade does not fairly represent their input to the group work. Each student will now have scores allocated to them by the other members of their group. This peer scoring is analysed to identify any unusual mark distributions. The median and mean of a student’s score are determined and subtracted from one another. Normally the median value is taken as the student’s final peer mark. If the difference between the median and the mean is greater than 2 (or -2) the distribution is investigated by a member of staff as this indicates that there may be an outlier. In this instance if there is one identifiable high or low score that does not match the other scores it is deleted and the resulting median value is taken as the student’s final peer mark. If there is more than 1 outlier, no score is deleted and the mean value is taken as the final student peer mark. The total peer marks for each group is obtained by adding the medians of all the group members, unless they have been awarded a mean as described earlier, then the mean is added instead. Then the fraction of the total work applicable to each individual student is obtained by dividing their individual peer mark (median or mean) by the total peer marks. This value is then multiplied by the total marks available from the staff marking calculated as already described, to give a mark out of 100%. This scheme allows a student, if awarded particularly high scores by their fellow group members, i.e. identified as having carried out the majority of the work, to end up with a score of more than 100%. In the past a constraint was put that nobody could have over 100%. Two years ago, after discussion with the external examiner for the course, it was decided that as this process allows students to reward fellow students for their contribution to the work the grades should stand. Consequently we have had the situation where a couple of students each year have been awarded more than the 10% the assessment is worth.  Each year several students have queried their individual scores. These queries are dealt with by the course coordinator. In all cases, after discussion with the student, investigation of the Moodle forums and looking at the group mark distributions, the initial score was justified and explainable. Current Situation: The system is currently working well. It is important to emphasise that this system has proved successful for such a large class size due to the assistance of a dedicated systems analyst and the use of suitable technology. The programming required for generating the webforms and using Excel to calculate the marks is not extremely advanced and can be accomplished in a number of ways, but it does require someone with suitable experience. However it would be perfectly feasible to run a similar peer-assessment scheme in a small class without needing the technology and just using paper forms. With several years refining and developing this peer- assessment system, responding to student and staff suggestions, we feel that we now have a system in place that, with minor updating each year, does exactly what we wished of it. The system allows students to meaningfully and appropriately award their peers for their contributions to a piece of group work. They benefit by taking more responsibility for their learning and appreciate being involved in their own assessment. The system has proved to be adaptable and is now used for similar group work tasks of presentations, posters and debates across a range of courses at different levels here at the University of Glasgow.  

Peer-assessment of group work in a large class: development of a staff and student friendly system

Enlighten: Publications

  
 
 
 
 
 
 
 
 
Cogdell, Barbara, Finlay, Chris, and McVey, Mary (2011) Peer-assessment 
of group work in a large class: development of a staff and student friendly 
system. In: Orsmond, Paul (ed.) Self- and Peer-Assessment: Guidance on 
Practice in the Biosciences. Series: Teaching Bioscience: Enhancing 
Learning Series . The Higher Education Academy, UK Centre for 
Bioscience,, Leeds, pp. 49-51. ISBN 9780956928832 
 
 
Copyright © 2011 The Higher Education Academy  
 
A copy can be downloaded for personal non-commercial research or study, 
without prior permission or charge 
 
Content must not be changed in any way or reproduced in any format 
or medium without the formal permission of the copyright holder(s) 
 
 
When referring to this work, full bibliographic details must be given 
 
 
 
http://eprints.gla.ac.uk/90943/ 
 
 
 
  Deposited on: 07 March 2014 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Enlighten – Research publications by members of the University of Glasgow 
http://eprints.gla.ac.uk 
Background and Rationale: 
First year biology at the University of Glasgow is divided into two sequential courses, which both 
have a group project assignment worth 10% of the summative assessment. We decided to use peer 
assessment for these group assignments, not only to cope with the fact that it would be impossible 
for the staff to directly monitor the activity of every group, but also to enhance the student learning 
experience.  
There are between 600 and 700 students taking each course. They are divided into a maximum of 16 
laboratory classes with about 48 students in each. Each laboratory class is then further divided into 
six groups of eight students, giving a possible maximum of 96 groups. The students are timetabled 
for specific laboratory times, and have been allocated specific laboratory seats and so become 
familiar with one another from the beginning of the academic session. 
The first assignment is based on ‘Darwin’s Dilemma’ and requires students (in their groups of up to 
eight) to prepare a 5 minute presentation arguing the case for the elimination of an invasive 
organism from a chosen geographical area. The presenting group are then questioned by an ‘expert 
panel’ who will test their knowledge of the particular organism. This expert panel is made up of 
fellow students from another group. Consequently every student group will act as a presenting 
group and as an expert panel for another group. A similar format is repeated for the second group 
assignment, which is to select an important biological molecule and construct a poster showing its 
structure; how and where it is produced; and its biological role. The task is completed by a 5 minute 
presentation defending the poster’s content with questions from an expert panel as before. The 
groups meet both in scheduled laboratory classes and in their own study time to research their 
chosen topics and complete their tasks. 
Though the scheduled laboratory classes are managed by members of staff, the staff cannot easily 
monitor the process when the students meet outside of their timetabled classes. Several tools are 
used both to encourage the students to monitor themselves and to allow staff to ‘follow’ the 
progress of the group work. Each group is required to elect a group leader and deputy. These 
individuals’ details are recorded for each group. The leaders are asked to note any relevant factors 
(e.g. people who seldom attend meetings; people who were helpful in researching the topic but not 
present at the presentation, etc.) as they may be consulted in the event of erratic peer marking. In 
order to assist with the communication within a group staff set up a specific Moodle forum for each 
group. Moodle is the VLE used at the University of Glasgow. These specific forums allow group 
members to discuss their work with each other without allowing students from other groups to see 
their ‘posts’. However staff can see all the ‘posts’ and can use these as a way of monitoring group 
progression. A series of deadlines for the group task are posted on Moodle. Each group is required 
to ‘post’ the division of responsibilities of its members. All students are made aware of the peer-
marking aspect of the assignments and each group is asked to ‘post’ their agreed marking criteria 
onto their forum. This can then be referred to by staff if required. The marking criteria will vary 
across groups and usually include points such as attendance at group meetings. 
There is a staff marking element to the groups’ presentations and expert panel questioning. Two 
members of staff are present for each presentation and agree a score for the group’s presented 
information and their ability to answer to questions from the expert panel.  The criteria that staff 
use for these scores, e.g. audibility, links to research, time keeping, etc., are made available to 
students several weeks before the presentations themselves. 
This staff mark is only part of the scoring used to generate a final, student specific grade. As a result 
of previous experience with group work it was also felt to be important to introduce a method of 
distinguishing individual contributions, i.e. to introduce peer-assessment. In an original group work 
activity there had been continuous complaints that the staff did not assess the group work and that 
the group members did not like carrying non-contributors. In the last couple of years of the original 
version of group work a high proportion of the students failed to contribute to the group activity. 
Development Through Experience: 
For the first introduction of peer-assessment each group was awarded a staff mark out of 100% for 
their work. This mark was multiplied by the number of members in the group to give the total 
number of marks that would be distributed across the group. The groups were then made aware of 
this total score and asked to divide the marks between themselves. For example, if a group has eight 
members and receives a staff score of 60% the total number of marks to be distributed across the 
group members is 480. 
The students allocated their peer marks together as a group in a scheduled laboratory session. Each 
group was given a single form with the full names and student ID numbers of each of their fellow 
members and a space to indicate their marks. These completed sheets were collected by the staff 
member leading the laboratory. It was noted that many groups agreed to share the marks equally 
amongst themselves. 
This scheme was fairly simple to run as there was an agreed sheet of marks per group and staff 
subsequently input these agreed marks into the assessment spreadsheet. Checks were made to 
ensure that the students had made correct calculations. Any queries could be sorted by consultation 
with the group leader and any students awarded zero by their group were investigated by staff for 
any extenuating circumstances.  
However the students did not like this scheme. Allocating marks in a group setting made it difficult 
and awkward to award low marks. Consequently non-contributors would get the same marks as 
everyone else and group members often felt resentful. 
Some groups will always be dysfunctional and as the mark contributes to their final course grade, it 
is unfair to tell the students to sort this out by themselves. A lot of additional staff work was 
generated trying to address issues like this. 
Next Step – Make the marking confidential and automate the marks calculation: 
The department’s systems analyst devised forms using Microsoft Word with mail merge, which could 
be fed into an Intelligent Character Recognition (ICR) machine. This machine reads paper forms with 
text entries electronically and the scores may then be transferred directly into an Excel spreadsheet. 
All subsequent calculations can be completed on the spreadsheet. Using these forms made it 
feasible for each student to submit their marks confidentially. 
As before the group work was awarded a staff score out of 100%. Again this score was multiplied by 
the number of students to give the total number of group marks for distribution. Students submitted 
their individual scoring forms, marking each of their fellow group members out of 10. These forms 
were read electronically and an average peer mark (out of 10) was calculated for each student. All 
the average marks for the group were added together to give a sum of peer marks for the whole 
group. This was then used to calculate the proportion of the total peer marks that should be 
awarded to each individual. Although it is possible to get over 100% with this formula final marks 
were capped at 100%. A major advantage of this system is that it takes into account whether the 
students are harsh or lenient markers. 
Obviously the system will not work if students fail to return their forms and so students were 
informed that failure to submit their scores would result in no marks being awarded for their work. 
The students were much happier with this system and appreciated being able to reward hard work 
and penalise freeloaders. There were favourable reports from both the end-of-course 
questionnaires and staff-student liaison meetings. Some students with low grades did complain, but 
because group leaders had been instructed to note attendance at group meetings it was relatively 
easy to point out to them that they had contributed very little and they usually agreed without 
further complaint. 
This second iteration solved the student identified issues, but still involved a major administrative 
task for staff. The ICR system is very efficient at reading the student forms, but relies on students 
using legible script and filling all boxes correctly. Each time the forms were illegible or filled in 
wrongly they had to be checked by the operator. With the large student numbers involved this 
became very onerous. 
Latest Development – Move the submission form and admin online: 
In an attempt to reduce staff time the system has developed to include online administration and 
handling of student scoring. Now the students are required to enter their marks for their colleagues 
using a web-based form. Each student is e-mailed a unique URL code, which is generated from their 
student ID number. This URL allows students to access their own specific webpage with a webform 
with a pre-generated list of their other group members and spaces to enter their marks. The 
webform asks each student to distribute 100 marks across the other members of their group. The 
webform has built-in validation and so cannot be submitted with any blank fields or if the score 
allocation does not add up to 100. 
In addition, each student is also asked to award themselves a mark out of 10 and include a 
paragraph explaining their self awarded score. This score is not used in the final allocation of marks 
but can be referred to, if a student feels that their final grade does not fairly represent their input to 
the group work. 
Each student will now have scores allocated to them by the other members of their group. This peer 
scoring is analysed to identify any unusual mark distributions. The median and mean of a student’s 
score are determined and subtracted from one another. Normally the median value is taken as the 
student’s final peer mark. If the difference between the median and the mean is greater than 2 (or -
2) the distribution is investigated by a member of staff as this indicates that there may be an outlier. 
In this instance if there is one identifiable high or low score that does not match the other scores it is 
deleted and the resulting median value is taken as the student’s final peer mark. If there is more 
than 1 outlier, no score is deleted and the mean value is taken as the final student peer mark. 
The total peer marks for each group is obtained by adding the medians of all the group members, 
unless they have been awarded a mean as described earlier, then the mean is added instead. Then 
the fraction of the total work applicable to each individual student is obtained by dividing their 
individual peer mark (median or mean) by the total peer marks. This value is then multiplied by the 
total marks available from the staff marking calculated as already described, to give a mark out of 
100%. 
This scheme allows a student, if awarded particularly high scores by their fellow group members, i.e. 
identified as having carried out the majority of the work, to end up with a score of more than 100%. 
In the past a constraint was put that nobody could have over 100%. Two years ago, after discussion 
with the external examiner for the course, it was decided that as this process allows students to 
reward fellow students for their contribution to the work the grades should stand. Consequently we 
have had the situation where a couple of students each year have been awarded more than the 10% 
the assessment is worth.  
Each year several students have queried their individual scores. These queries are dealt with by the 
course coordinator. In all cases, after discussion with the student, investigation of the Moodle 
forums and looking at the group mark distributions, the initial score was justified and explainable. 
Current Situation: 
The system is currently working well. It is important to emphasise that this system has proved 
successful for such a large class size due to the assistance of a dedicated systems analyst and the use 
of suitable technology. The programming required for generating the webforms and using Excel to 
calculate the marks is not extremely advanced and can be accomplished in a number of ways, but it 
does require someone with suitable experience. However it would be perfectly feasible to run a 
similar peer-assessment scheme in a small class without needing the technology and just using paper 
forms. 
With several years refining and developing this peer- assessment system, responding to student and 
staff suggestions, we feel that we now have a system in place that, with minor updating each year, 
does exactly what we wished of it. The system allows students to meaningfully and appropriately 
award their peers for their contributions to a piece of group work. They benefit by taking more 
responsibility for their learning and appreciate being involved in their own assessment. 
The system has proved to be adaptable and is now used for similar group work tasks of 
presentations, posters and debates across a range of courses at different levels here at the 
University of Glasgow. 
 


http://eprints.gla.ac.uk/91943/1/91943.pdf

Peer-assessment of group work in a large class: development of a staff and student friendly system

Abstract

Similar works

Full text

Available Versions

Enlighten

Enlighten: Publications