Broadfoot, Patricia

Oldfield, Alison

Sutherland, Rosamund

Timmis, Sue

English

Explore Bristol Research

Timmis, S., Oldfield, A., Sutherland, R., & Broadfoot, P. (2014).Exploiting the collaborative potential of technology enhancedassessment in higher education. University of Bristol.http://www.bristol.ac.uk/education/people/rosamund-j-sutherland/pub/32067459Publisher's PDF, also known as Version of recordLink to publication record on the Bristol Research PortalPDF-documentUniversity of Bristol – Bristol Research PortalGeneral rightsThis document is made available in accordance with publisher policies. Please cite only thepublished version using the reference above. Full terms of use are available:http://www.bristol.ac.uk/red/research-policy/pure/user-guides/brp-terms/Exploiting the collaborative  potential of technology enhanced assessment in higher education Rethinking  AssessmentDiscussion paper 3:Q: How can digital technologies enhance peer assessment in higher education?Q: What are the key challenges for technology enhanced peer assessment?This paper considers the following aspects  of technology enhanced peer assessment: The rise of peer assessment in  higher education Benefits and challenges of peer assessment Peer assessment using technology Peer assessment and disciplinary culturesKey recommendations Policy makers and practitioners should acknowledge the importance of peer collaboration and networks for learning and recognise that learning is social, distributed and collective.  Successful peer assessment requires individual responsibility from students, interdependence on peers, and trust within groups. Practitioners should recognise that students can be anxious about the ability of their peers to assess learning, their own abilities to assess others’ work and the overall validity of peer assessment. Digital technologies have the potential to support collaborative learning and assessment practices, such as undertaking knowledge building activities, co-evaluation and social interaction.The rise of peer assessment in higher educationInterest in – and the practice of – peer assessment is growing in higher education (HE). A type of collaborative learning activity in which students assess and feedback on the work of their peers (peer assessment ) has a wide range of potential uses in HE. It can contribute to students’ marks, provide formative feedback, deliver more equitable methods of assessing group work contributions, and support blogs or portfolios for professional degrees or practical work. Peer assessment is thus often recognised as a complex form of assessment that can support a wide range of learning outcomes. Research evidence points to the importance of peer collaboration and networks for learning and emphasises that learning is ‘social, distributed and collective’.1 Thus, learning is not a passive or solo venture but is active, social, contextual and situated in real-world living. The involvement of the wider learning community in assessment makes sense when learning is no longer viewed as an individual activity. Additionally, as HE emphasises skills that prepare students for professional work contexts – such as peer learning and team work – assessment should reflect those priorities. It is in this context that peer assessment emerges as a valuable way to provide participatory, active and social feedback. Benefits and challenges of peer assessmentPeer assessment techniques vary greatly across institutions and among different HE disciplines. Likewise, the many advantages that peer assessment can offer range from positive impacts on the quality of learning processes and outcomes, improved social interaction and enhancement of metacognitive skills. Specific notable benefits include:  Developing critical reflection skills and deeper understanding of course content and assignment criteria  Enhancing the quality of learning and ownership over one’s own work  Improving students’ performance in areas such as writing  Increasing students’ enjoyment of learning Decentralising the locus of control on assessment and moving the focus of power and judgment closer to the students 2However, the practices of peer assessment also bring their own challenges. Peer assessment often uses questionnaires or numerical scales to rank, nominate or rate peer performances. Such techniques can provoke adverse reactions from students or be seen as biased, for example in ‘overmarking’ (also known as ‘friendship marking’) or ‘decibel marking’ (the dominant voices in a group get the highest marks).3 Recorded student perceptions of peer assessment also demonstrate that peer learning invokes emotional challenges. Successful peer assessment requires individual responsibility from students, interdependence on peers, and trust within groups.3 Despite evidence that peer marks on average agree with teacher marks, students can express anxiety about the ability of their peers to assess learning, their own abilities to assess others’ work and the overall validity of peer assessment.4 Such findings illustrate that peer assessment is a complex skill that requires training and support for students, both in giving feedback and in receiving and managing evaluations of their own work. Lastly, HE institutions may not recognise the benefits of peer interaction, whose processes also do not integrate well with HE cultural norms that emphasise individual work, often viewing collaboration as being aligned with collusion or even plagiarism. Indeed, some HE institutions have policies that, in an effort to be fair and equitable, ‘actively mitigate against such reflection and dialogue’.5 Peer assessment using technologyDigital technologies have the potential to support collaborative learning and assessment practices, such as undertaking knowledge building activities, co-evaluation and social interaction.6 Employment of digital technologies to support collaboration and peer learning has been particularly notable in the field of computer-supported collaborative learning (CSCL), which investigates collaborative enquiry using technology. However, little emphasis has been placed on how to assess such collaboration or evaluate individual contributions to collective learning. One study assessed a collective knowledge building activity using portfolios, through which students evaluated both individual and collective learning via self- and peer assessment.1 Resulting recommendations for assessing collaborative learning included creating cultures that emphasise collaboration over individual competition and developing students’ agency and ownership over their own and their peers’ assessments. Additionally, the use of digital technologies does not guarantee enhanced collaboration, and CSCL activities do not always support equal opportunities for participation, interaction or greater ownership over learning processes. For example, a case study using peer assessment in a CSCL environment found limited participation of students in some assignments and generally low-quality assessment reports.3Particular types of digital tools lend themselves well to peer assessment practices. One recent study used electronic voting systems (EVS)7 to help students work together and engage deeply with assessment criteria.8 Working in groups, students used EVS to evaluate previous students’ assignments according to specified marking criteria. The study resulted in improved quality of students’ practical work and a significant improvement in the types of discussions around assessment practices.Web 2.0 tools such as wikis, blogs and social networks can also support collaboration and increased participation in teaching and learning processes. These tools have been linked to increases in self, peer and group work assessment,9 but a 2009 JISC review of e-assessment techniques found little evidence of these tools actually being used in current assessment practices.10 The transformative potential of Web 2.0 technologies has also been questioned in relation to social and educational identities and inequalities. Some argue that using tools like wikis or blogs can further exclude some students by benefitting those who are already users of social media.11 Finally, the use of digital technologies can also address challenges of peer assessment noted above. One study attempted to address ‘peer bias’, or the allocation of positive assessments based on friendship or personality.12 To do so, it employed EVS to anonymously evaluate student work based on specific marking criteria. Despite students reporting generally positive experience in the study, they resisted the idea that peer evaluation should become a formal component of their marking. Additionally, students’ familiarity with such voting techniques in mainstream television talent shows actually increased their anxiety of being judged by peers, despite the anonymity the technology provided.Peer assessment and disciplinary culturesThough peer assessment practices are derived from developments in social learning theories and current understandings of feedback processes, they remain relatively uncommon assessment techniques. This is due to a number of factors across HE environments. At a classroom level, assessment of collaborative learning activities is poorly understood and evaluation still primarily involves teachers or lecturers controlling tasks and assessment. At a wider institutional level, peer interaction and learning in communities often takes a back seat to the priorities of personalisation and individual learning.13 Peer assessment challenges these patterns through its involvement and ownership of tasks by students. In order to facilitate a broader adoption of peer assessment practices, a wider cultural shift in HE assessment is required. In line with prevailing learning theories, assessment practices should challenge the current emphasis on individual learning and promote collaboration. This ultimately requires a deeper institutional understanding of and commitment to the benefits of peer learning and assessment, as demonstrated through policies and supported practices. To support such a shift, peer assessment practices and experiences should be more visibly and widely shared among educators. This would clarify peer assessment’s benefits, elucidate the methods that elicit these advantages and share the required time investment to make peer assessment happen. This is particularly important to consider across disciplines, due to the diversity of the ‘ways of thinking and practicing’ that manifest in different HE disciplinary cultures.14 1 van Aalst, J., & Chan, C. K. K. (2007) ‘Student-directed assessment of knowledge building using electronic portfolios’. Journal of the Learning Sciences, 16(2), pp. 175–220.2 Bennett, S. and Barker T. (2012) ‘The use of electronic voting and peer assessment to encourage the development of higher order thinking skills in learners’. International Journal of e-Assessment, 2 (1).  Cartney, P. (2010) ‘Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used’. Assessment & Evaluation in Higher Education, 35 (5), pp 551–564. Van den Berg, I., Admiraal, W. and Pilot, A (2006). Peer assessment in university teaching: evaluating seven course designs, Assessment and Evaluation in Higher Education, 31(1), 19-36 Grajczonek, J. (2009) ‘Exploring Students’ Perceptions of Peer Assessment in Group Work for Allocation of Individual Marks in Higher Education’. International Journal of Learning, 16 (3), pp. 105-125.3 Prins, F.J., Sluijsmans, D.M.A., Kirschner, P.A. and Strijbos, J. (2005) ‘Formative peer assessment in a CSCL environment: a case study’ Assessment & Evaluation in Higher Education. 30 (4), pp. 417–444.4 Falchikov, N. and Goldfinch, J. (2000) ‘Student peer assessment in higher education: A metaanalysis comparing peer and teacher marks’. Review of Educational Research. 70 (3), pp. 287 – 323; Cartney, P. (2010) ‘Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used’. Assessment & Evaluation in Higher Education, 35 (5), pp 551–564.5 Ferrell (2012) A view of the Assessment and Feedback Landscape: baseline analysis of policy and practice from the JISC Assessment & Feedback programme. A JISC report, p. 9. jisc.ac.uk6 See bristol.ac.uk/education/research/sites/tea7 Electronic voting systems, also known as audience response systems, are tools that support interaction between audiences and a presenter. In education, this usually happens through the use of wireless remote controls that students use to respond to questions or prompts provided by a lecturer.8 Bennett, S. and Barker T. (2012) ‘The use of electronic voting and peer assessment to encourage the development of higher order thinking skills in learners’. International Journal of e-Assessment, 2 (1). 9 Whitelock, D. (2010) ‘Activating Assessment for Learning: are we on the way with Web 2.0?’ In Lee, M.J.W. and McLoughlin, C. (Eds.) Web 2.0-Based-E-Learning: Applying Social Informatics for Tertiary Teaching. IGI Global. pp. 319–342.10 Ripley, M., Tafler, J., Ridgway, J., Harding, R. and Redif, H. (2009) Review of Advanced e-Assessment Techniques. A JISC report: jisc.ac.uk11 Hughes, G. (2009) ‘Social software: new opportunities for challenging social inequalities in learning?’ Learning, Media and Technology, 34 (4), pp. 291-305.12 Walker, R. and Barwell, G. (2009) ‘Click or clique? Using Educational Technology to Address Students’ Anxieties About Peer Evaluation’. International Journal for the Scholarship of Teaching and Learning, 3 (1), pp. 1-20.13 Timmis, S. (2012) ‘Constant companions: Instant messaging conversations as sustainable supportive study structures amongst undergraduate peers’. Computers and Education, 59, pp. 3-18.14 Entwistle, N. (2005) ‘Learning Outcomes and Ways of Thinking across Contrasting Disciplines and Settings in Higher Education’. Curriculum Journal, 16 (1), pp. 67-82.Learning is not a passive or solo venture but is active, social, contextual and situated in real-world livingAssessment practices should challenge the current emphasis on individual learning and promote collaborationCase study: PEER: Peer Evaluation in Education Review(reap.ac.uk/PEER.aspx) PEER is part of the REAP initiative ‘Re-Engineering Assessment Practices’ operating since 2005. The PEER project is exploring ways of harnessing technology to make peer review easy and cost-effective to implement. It aims to show that learning is significantly enhanced when students are involved in making judgements and giving feedback on the work of peers. Pilots indicate many assessment-related benefits such as revisions, discerning levels of detail required and developing concise answers. In some subjects, students demonstrated a lack of experience of this kind of activity. Improvements often included ensuring that students were sufficiently supported in undertaking peer reviews. Rethinking Assessment2012/2013 Series of discussion papers Assessment is universally recognised as one of the most important – and powerful – elements of an educational experience. It is also seen as one of the hardest to reform. However, there is an increasingly accepted need for rethinking assessment if it is to keep up with current theoretical, cultural and technological developments affecting teaching and learning. Digital technologies open up new possibilities for more personalised, immediate and engaging assessment experiences. However, the use of digital technologies for assessment (referred to as ‘technology-enhanced assessment’) has yet to be ‘transformative’, with current practices either replicating traditional assessment methods or manifesting in pockets of innovation that are not widespread. How the potential of digital technologies can best support improved assessment practices and preferred educational outcomes is becoming an issue of increasing importance. An acknowledgement of the potential that digital technologies offer should recognise the complexity of the task, the many factors affecting successful educational change, and the significant ethical questions raised by the use of digital technologies in assessment. This series of discussion papers draw on a substantial review of literature which aimed to identify the different ways in which technology currently impacts on educational assessment practices and how it could contribute to a new vision for assessment.  The review of literature is available at:  bristol.ac.uk/education/research/sites/teaThe following discussion papers have been produced in order to highlight key issues  and questions identified by the review of literature:Paper 1: Transforming education through technology enhanced assessmentPaper 2: Integrating the formative and summative through technology  enhanced assessmentPaper 3: Exploiting the collaborative potential of technology enhanced  assessment in Higher EducationPaper 4: Learning analytics and technology enhanced assessmentPaper 5: Ethical issues in technology enhanced assessmentPaper 6: National standards and technology enhanced assessmentSeries authors:Patricia Broadfoot, Sue Timmis, Sarah Payton, Alison Oldfield, Rosamund SutherlandFor further information contact Sue Timmis at Sue.timmis@bristol.ac.ukbristol.ac.uk/education3. Exploiting the collaborative potential of technology enhanced assessment in higher education Graduate School of EducationUniversity of BristolHelen Wodehouse Building35 Berkeley SquareBristol BS8 1JATel: +44 (0)117 928 9000STELLARDESIGN: CW-DESIGN.CO.UKSupported by:

Exploiting the collaborative potential of technology enhanced assessment in higher education

https://research-information.bris.ac.uk/ws/files/135276385/collaborating.pdf

Exploiting the collaborative potential of technology enhanced assessment in higher education

Abstract

Similar works

Full text

Available Versions

Explore Bristol Research