Coleman, Sonya

Kerr, Dermot

English

Group projects are an important part of undergraduate computer science learning because of their role in developing working skills which are vital for professionals in the computing industry. While group projects offer many potential learning benefits, there is no guarantee that the development of working skills will be achieved. In fact, group projects introduce their own stresses and strains for students due to the need to share the workload as fairly as possible, in how individual contributions are measured and recognised, the effect this has on individual performance, and ultimately how this contributes to the student’s success in the course. Group projects which are not designed, supervised and assessed in a way that promotes meaningful teamwork and collaboration can lead to failure. One approach to supporting group work and encouraging engagement by all is to use self and peer assessment. The concept of peer-assessment (also known as peer review) has been used in education and beyond for many years. It provides a transparent mechanism through which students can critique and provide summative feedback to their fellow students on their work. Additionally, this may encourage students to take an independent approach to their own personal learning. For example, self and peer assessment encourages students to engage with the assessment criteria and also to reflect on their own performance as well as the performance of their fellow students. This is considered by many as an important process as developing and providing peer feedback to other students also assists students in developing critical thinking skills and objective evaluative judgements based on the assignment criteria. With group projects students are often disgruntled by the idea that all group members will receive the same mark regardless of individual effort. In this paper we demonstrate practical use of the WebPA system to allow students to perform self- and peer-assessment to effectively measure individual contributions within group projects and improve the student experience. The impact of the tools in supporting and measuring performance is validated through quantitative and qualitative student feedback where we demonstrate two key points: firstly, the students generally had an enjoyable experience in the assessment process and secondly, we achieved student satisfaction in the mark allocation process

Ulster University's Research Portal

IMPROVING THE STUDENT EXPERIENCE: SELF- AND PEER-ASSESSMENT IN GROUP PROJECTS  Dermot Kerr, Sonya Coleman Ulster University, Northland Road, Londonderry, BT 48 7JL, United Kingdom  Abstract Group projects are an important part of undergraduate computer science learning because of their role in developing working skills which are vital for professionals in the computing industry. While group projects offer many potential learning benefits, there is no guarantee that the development of working skills will be achieved. In fact, group projects introduce their own stresses and strains for students due to the need to share the workload as fairly as possible, in how individual contributions are measured and recognised, the effect this has on individual performance, and ultimately how this contributes to the student’s success in the course. Group projects which are not designed, supervised and assessed in a way that promotes meaningful teamwork and collaboration can lead to failure.  One approach to supporting group work and encouraging engagement by all is to use self and peer assessment. The concept of peer-assessment (also known as peer review) has been used in education and beyond for many years. It provides a transparent mechanism through which students can critique and provide summative feedback to their fellow students on their work. Additionally, this may encourage students to take an independent approach to their own personal learning. For example, self and peer assessment encourages students to engage with the assessment criteria and also to reflect on their own performance as well as the performance of their fellow students. This is considered by many as an important process as developing and providing peer feedback to other students also assists students in developing critical thinking skills and objective evaluative judgements based on the assignment criteria.  With group projects students are often disgruntled by the idea that all group members will receive the same mark regardless of individual effort. In this paper we demonstrate practical use of the WebPA system to allow students to perform self- and peer-assessment to effectively measure individual contributions within group projects and improve the student experience. The impact of the tools in supporting and measuring performance is validated through quantitative and qualitative student feedback where we demonstrate two key points: firstly,  the students generally had an enjoyable experience in the assessment process and secondly, we achieved student satisfaction in the mark allocation process. Keywords: Marks allocation, Peer-Assessment, Self-Assessment, Group projects, Improved Student Experience. 1 INTRODUCTION Peer-assessment (also known as peer review) is an assessment technique which has been used in education and beyond for many years. It provides a transparent process through which students can review, critique and provide summative feedback to their fellow students on their work. This can encourage students to take an independent and personal approach to their own learning. For example, peer and self assessment encourages students to fully engage with the criteria used for assessment and also encourages students to reflect on their own contributions as well as the contributions of their fellow students. Reflective practice is considered by many as an important skill for developing and providing peer feedback to other students and also assists students in developing objective evaluative judgements and critical thinking skills based on the assignment criteria. Beyond this, it provides students with the means to develop lifelong transferable skills for assessing and providing feedback to others in all aspects of life.  Moreover, it equips students with the tools and self-awareness to self-assess and improve their own work. The UK Higher Education Academy (HEA) presents a paper on peer assessment that notes that  the most difficult aspect of group assessments is the ability to assess the group work as a whole and also the individual contribution of each group member to the overall product to be assessed [8]. This is a key problem that results in students being resentful of group work as they feel that perhaps they are contributing more to the assessment than others and yet that is not reflected in the corresponding mark received. From a pedagogic perspective, this is unacceptable in terms of quality assurance. Therefore, peer assessment has been the focus of much pedagogic research, for example [2], where marks or weightings assigned by individual team members to their fellow team members are used to modify a team mark allocated by the project supervisor through various mechanisms [6] and thus obtain an individual mark associated with perceived contribution. Similarly, peer assessment comes with its own challenges, both in terms of giving and receiving peer feedback. Student perceptions of work will differ, and some will naturally be severe in their assessments and other students may respond badly to that, often causing them to question the value of their work. Additionally, students can be wrong when it comes to the assessment of another's work, particularly if they are inexperienced in the area of peer assessment or the topic in question.  Therefore, this should be overseen by a lecturer or person responsible for the course delivery.  However, one of the advantages of peer assessment is that it provides deep learning to students as well as an appreciation of the assessment process.   To permit students to properly assess the work of their peers, they need to acquire a thorough understanding of the assessment criteria to be used and the expectations of the assignment itself; this naturally encourages deeper learning [5].  Consequently, deep learning and self-evaluation may significantly impact on a students overall assessment performance [1, 3]. Peer assessment also has obvious benefits for teaching staff.  It has the potential to reduce the assessment marking workload by transferring some responsibility to the students and this naturally helps to alleviate the pressures associated with large classes and other difficulties. However, it is vital that significant thought and effort is allocated when designing the peer assessment; for peer assessment to work successfully, students must understand what it is, why it’s being carried out and how it will work. Thus, this paper describes an approach to automated peer marking for a group assessment in an undergraduate level User Experience (UX) module using the Web-PA platform.  We explore the students’ acceptance of peer marking to determine final group mark allocation, the pedagogic issues associated with it and reflect on the general impact on module marks.  2 METHODOLOGY The study is based on a group project for a mixed cohort of undergraduate students studying Computer Science, Software Engineering and Information Technology. All students were enrolled on a 12-week second year module on the topic of UX and user interface design. For the assessment students were divided into groups of four through a process of student self-enrollment, with the remaining students assigned by the lecturer. In the study presented here there were 120 students in the cohort resulting in 30 student groups. The groups were tasked to develop a prototype front-end user interface to solve a real-world computing problem using one of ten provided scenarios. Each scenario asked the students to develop a web-based user interface for a desktop or mobile device. The overall aim of the group project was to develop core competency and confidence in client-side web development while appreciating the need for user-centred design [7]. Essentially, this requires the students to understand the intended system users, their needs and requirements, while reflecting on the context of their actions - only after this has been completed can the actual system design be considered in terms of conceptual and physical design as the design should be based on real people, with real tasks and needs. User-centred design is a hands-on design process that requires student groups to identify actual users (stakeholders), conduct a focussed discussion on the tasks they are trying to complete, and thoroughly understand the context and environmental constraints of their work. Students will then develop their initial user interface designs on the outcome of such discussions although in many cases the initial designs are likely to be crude and error prone. The expectation is that students will work in groups, discussing and identifying potential user-related issues, using a strategy that enables continuous evaluation and refinement of the proposed design. Group work is an essential aspect of this process as students need to appreciate both user and business needs and apply and reflect on user-centred design techniques as part of their learning experience. Collaborative interaction ensures the development of transferable skills that are essential in employment, such as compromise, cooperation, negotiation, delegation, etc. Students also benefit by learning from each other. Students have different expectations, views, ideas and contributions on the requirements, design and implementation stages; this combined input helps students appreciate user-centred design. Once students have completed their projects they are subsequently tasked to self- and peer-review the individual members of their group using 5-point criteria that was defined by the students during the semester.  2.1. Web based peer review The use of a single mark to grade a group project is often a worry for many undergraduate students. Students often perceive, rightly or wrongly, that individuals working on the project are not given the credit they deserve for their contributions, and group members with minor or no contribution receive the same marks as those who have contributed to the project in a  major way. Peer assessment permits better grading of a student's abilities against a range of important transferable skills that they will have to demonstrate as graduates, such as communication, leadership, presentation skills, working as part of a team, etc. Completing the peer-assessment is a key aspect of learning as it ensures students reflect and assess their group peers’ abilities as well as their own abilities.  The selected peer assessment system, Web-PA, was developed in Loughborough University as a web-based system for both self and peer assessment [9].  It has been used across a number of faculties at Loughborough for several years. There is some early evidence of higher levels of student satisfaction with respect to group work since Web-PA was introduced [4]. Web-PA can handle large classes of students and has been previously described in detail by the authors [9]. It provides an easy way by which individuals can be assigned a proportion of the overall project mark based on information provided by individual members of the groups. Web-PA allows the teaching staff to establish a number of criteria against which each group member marks themselves (self) and each of the other group members (peers). This is typically completed at the end of a project but could also be carried out at prescribed times throughout the project lifetime. Students enter data in the confidence that other members of the group cannot see their choices and the system then calculates a variation factor (called the Web-PA factor) for each individual group member based on the normalised total score (an example of this is illustrated in Figure 1 where we can see that, for overall contribution to the project, Student B has awarded all students (including themselves) a maximum or 5 marks.  However Student C has awarded Student A 5-marks, Student B 3-marks, Student D 5-marks and only awarded 4-marks to themselves.   Figure 1: Example of the use of WebPA for peer assessment [10]  The teaching staff mark the group project submission in the normal manner considering the mark to be awarded for the complete overall submission. Then this mark, or part of it, is weighted by the calculated variation factor for each individual. In the case when all group members score equally, the Web-PA factor = 1.0, therefore all members achieve the unmodified team mark. In the case when all group members do not score equally the variation factor is used to adjust the group mark up or down based on lecturer defined parameters. A visualisation of the benefits of WebPA is presented in Figure 2 where we can see that in Figure 2(a) traditionally students in a group all received the same allocated mark.  However, Figure 2(b) illustrates that, with the use of WebPA, students within a single group can each receive individual adjusted marks based on the variation factor.   (a) Traditional marking approach  (b) WebPA based marking approach  Figure 2: A visualisation of the benefits of WebPA [10]  3 EVALUATION AND RESULTS A questionnaire was designed for the students with a purpose of gathering quantitative and qualitative feedback on the use of the Web-PA system for self- and peer-assessment.  In general, the students considered Web-PA a fair means of calculating variation factors for individual contributions.  In total 37 students completed the questionnaire, and the key results are presented in Figure 3.  Figure 3(a) illustrates that approximately 72% found the assessment to be fair and reasonable and we believe this satisfaction rate can be attributed to the use of Web-PA.  In Figure 3(b) we can see that >75% students strongly agree or agree that Web-PA was a good approach to peer assessment. It should also be noted that satisfaction can be anecdotally measured by general student satisfaction of the module during the staff-student consultative process; many less queries and issues with the group assessment were raised. As illustrated in Figure 3, the majority of students were supportive of the use of WebPA and felt it was a fair way to assess group work. This is what they had to say (comments are anonymised):  “Assessments were fair and organised well.”  “I liked the group work”  “The course overall was interesting and engaging, with group work furthering my experience in a team”  “I found that the practical work set us up better for the team assignment than other modules have in the past”  “Found the practicals useful especially as it involved engaging as a group.”  “I enjoyed the engagement with my group on the project”  “I liked the module, didn't feel overly stressful at times and being able to work in a group was useful to discuss any problems with it”    (a) “I found the assessment fair and reasonable”  (b) “I found the Web-PA assessment a good way to evaluate my own and others' contribution to the project”  Figure 3. Quantitative evaluation of survey responses to two categories   4 CONCLUSIONS When students contribute to their marking criteria research has shown that student learning is enhanced. The criteria used must be clear and transparent for students in order for them to assess the work of their peers. This process of peer and self assessment supports the students in learning to understand what is meant by high quality and they can also learn from their peers that there may be different approaches to successfully completing an assignment. WebPA is an excellent tool to facilitate self- and peer-assessment in large classes as the tool offers students privacy in their rating of individual contributions and automatically calculates weighted scores for the teacher saving time. This peer assessment was conducted with second year students as such skills will be transferable to their placement (in third year) and final year and will ideally lead to increased course engagement. Peer assessment can provide frequent and relevant formative feedback that students can act upon in a valuable way to enhance their learning and skills. Peer assessment is an excellent way to encourage students to actively participate in group activities and self assessment encourages students to reflect on their own participation and performance. As the students get the opportunity to mark each other and therefore be evaluated by others which prevents them from ‘hiding’ in a group. It also engages them to pro-actively compare their own performance and engagement with others. This increased engagement naturally improves the learning and teaching experience for the students, ensuring that they fulfil all module outcomes. The student feedback on this approach was very positive and the module marks and attendance illustrated an improvement over previous years. Therefore, this will be used in the UX module in the forthcoming year. REFERENCES [1] Brown, S. Rust, C. & Gibbs, G. (1994). Strategies for Diversifying Assessment in Higher Education. The Oxford Centre for Staff Development, Oxford. Oxonion Rewley Press.  [2] Crawford, A, R., Tennant, J., Choo, B., S., Wilson, D.A. (Ed). (2003) A Guide to Learning Engineering through Projects, FDTL(3) report 43/99 PBLE: Project Based Learning in Engineering edited works, Sept 2003.  [3] Falchikov, N. (2007). In: Rethinking Assessment in Higher Education (Boud, D. & Falchikov, N., eds.), pp. 128-144. Abingdon. Routledge. [4] Gatfield, T. (1999) Examining student satisfaction with group projects and peer assessment. Assessment and Evaluation in Higher Education, 24 (4), 365-377. [5] Hughes, I E (1995) Peer assessment of student practical reports and its influence on learning and skill acquisition. Capability, 1, 39-43 [6] Lejk, M. & Wyvill, M., (1996) A survey of methods of deriving individual grades from group assessments. Assessment and Evaluation in Higher Education, 21 (3), 267-281. [7] Monk, A., (2000), June. User-centred design. In International Conference on Home-Oriented Informatics and Telematics (pp. 181-190). Springer, Boston, MA. [8] Race P. (2001) A Briefing on Self, Peer and Group Assessment. Higher Education Academy, Assessment Series 9. [9] Willmot, P., Crawford, A., R. (2004) Online peer assessed marking of team projects. Proceedings of the International Conference on Engineering Education, ICEE 2004, Paper WA7/4, Gainesville, Florida, USA, October 2004, 1-7. [10] https://webpaproject.lboro.ac.uk/academic-guidance/effective-practice-using-webpa  

Improving The Student Experience: Self- And Peer-Assessment In Group Projects

https://pure.ulster.ac.uk/files/91198371/EDULEARN2021_Accepted.pdf

Improving The Student Experience: Self- And Peer-Assessment In Group Projects

Abstract

Similar works

Full text

Available Versions

Ulster University's Research Portal