Arico, Fabio

University of East Anglia digital repository

Promoting Active Learning Through Peer-Instruction and Self-Assessment: A Toolkit to Design, Support and Evaluate Teaching

Promoting Active Learning through Peer-Instruction and Self-Assessment: a toolkit to design, support, and evaluate teaching  Fabio Aricò, University of East Anglia  Introduction  Active learning is advocated in the current pedagogical debate as one of the most powerful pathways to student engagement. However, whilst it is relatively easy to promote the adoption of active learning pedagogies in small class environments, the challenge arises when dealing with large cohorts of students.   To tackle this issue, having students teaching other students might be the solution: everybody is engaged on the learning task, everybody has the opportunity to add to her own knowledge, and to develop core skills. In this article, I will describe how I successfully implemented the Peer-Instruction pedagogy (Mazur, 1997) in my teaching practice, and how my experience can be useful to other teachers, and educational developers across the sector. In addition, and in line with the revived debate on the Scholarship of Learning and Teaching, I will argue that not only is important to promote the adoption of active learning pedagogies, but also to devise opportune strategies to assess their effectiveness.   In the first part of the article I will describe the learning environment that I have developed, drawing from my personal experience. I will then focus on the implementation of the Peer-Instruction pedagogy, which I have modified to include a self-assessment component. A few words will also be spent on how learning technologies can support the implementation of active learning in large cohorts of students. In the second part of the article I will describe how the data collected from teaching sessions can be analysed to construct indicators of learning and teaching learning effectiveness, including the student’s voice. In the final part of the article I will reflect on ethical considerations related to the use of student data, on how my experience could be extended to different learning environments, and on the pitfalls to be kept on check.  The Learning Environment  I teach a large first-year compulsory module in Introductory Macroeconomics, for the School of Economics at the University of East Anglia. The number of students varies within 150-250, and my task is to endow a heterogeneous population of students with a core set of skills that they will use in more advanced modules later on. At the beginning of the module students’ skills can be very different, as roughly half of the class comes for an international background. Some students might have studied Economics prior to coming to university, while for others it is a genuine fresh start. The module articulates in: (i) lectures, where I present the learning material and discuss it with the class, (ii) seminars, where students work on a pre-assigned problem-sets, and come to class to discuss it with their peers and seminar leaders, and (iii) workshops, where students walk in to a problem set never seen before and work in teams to solve it. While seminars are organised as small-class activities and facilitated by my team of associate tutors, workshop are large-class events led by me, as the class is divided in only two large groups for which I repeat the same session twice through with intervals of two weeks. Seminars and workshops are always based on material taught in lectures beforehand and it is assumed that students have had chance to read it and, at least partially, process it prior to practicing on it in class. As I mentioned, promoting discussion and active learning in seminars is relatively easy, given the size of the classes. The real challenge is promoting active learning in workshops but, with the aid of Peer-Instruction, and the support of learning technologies, I have managed to win this challenge very effectively.  Peer-Instruction with Self-Assessment  At the beginning of each workshop session, students receive a sheet with a set of multiple-choice questions. Students are endowed with Student Response Systems (SRS), or clickers, which they can use to respond to the questions. The questions are displayed on the classroom’s screen through a PowerPoint presentation, enriched with an ad-on which enables me to collect students’ responses. Thus, for each multiple choice question, I follow a very precise algorithm:  a) Students are asked a question and respond to it using their SRS devices, individually and autonomously. The distribution of answers is not revealed to them. b) Students are asked to rate their confidence at mastering the skills needed to respond to the question correctly. In this case the distribution of answers is revealed, so that students (and myself) can gauge the temperature in the class, and get a feeling of how challenging the question was. c) The same identical question (from point (a)) is asked again. Students are then invited to discuss their opinions on the available options, comparing with each other. This is Peer-Instruction. Some students will try to convince others that they are right. Others will listen or share doubts during the discussion. By the end of this stage, all students are invited to provide a second individual response to the question asked. d) The distribution of answers is revealed, the correct answer is highlighted, as I proceed to discuss the solution and take any further questions about the task just completed. The distribution of answers comparing the first and the second round are also revealed to the students so that they can visually see the change in the distribution of responses. Generally this polarises onto the right answer.  The algorithm is repeated for all the questions that compose the problem set, which may vary between 6 and 10. The whole process takes a little more than an hours. I apply as much flexibility as I can to regulate the time invested in tackling each question. The software I use informs me about the number of responses coming in in real time; I also observe the student dynamics in the class, which allows me to decide on when is the right time to close each poll. My own addition to the original algorithm devised by Mazur (1997) consists of the self-assessment question that precedes Peer-Instruction. In my opinion, it is extremely important that students have the chance to reflect on what they are doing, and critically evaluate their skills in relation to the tasks that they are assigned. To this extent, active learning is not just based on the interaction generated by Peer-Instruction, but the active engagement that each student, even individually, exercises with respect to her own learning.  The role of Student Response Systems  Intelligent use of learning technologies is of course a catalyst to the promotion of active learning in the classroom. My teaching methodology would work very well even without the aid provided by SRS, but the opportunity to display student responses on the screen acts as a further incentive to engage on task, and provides useful information to the teacher on where to focus attention to maximise learning. The opportunity to see the positive effect that Peer-Instruction generates in the class is also a further motivator for students. Each student in my School is assigned with a clicker device, which s/he retains until the end of her/his third year. Each clicker is associated to a unique ID code, which appears on my computer when I download the reports from each teaching session. Thus, by matching clicker ID codes with student records, I can track the clicking activity of each student all across their first year of studies in my module, and I can correlate this activity to student demographics and background information.   I use TurningTechnologies SRS devices, in conjunction with a USB radio receiver that collects responses once plugged in any computer. The software that manages this process is TurningPoint, a TurningTechnologies freeware package that integrates with PowerPoint to create interactive slides able to display the distribution of answers. TurningPoint can also be used as a standalone piece of software, able to manage student polling independently from PowerPoint. The software also allows me to save the data generated from each clicking session, which can be converted in Excel format, or used to produce reports to be shared with the students. Since the beginning, I found it intuitive and easy to use. TurningTechnologies also provide support, case-studies, and organise an annual user’s conference to share and disseminate good practice. In an alternative to clickers, TurningTechnologies also produce an application downloadable on portable devices, so that student can even interact with their lecturers using their own mobile phones. In this case, individual application licenses should be purchased, and renewed every year.    Assessing the effectiveness of Peer-Instruction and Self-Assessment  Student Response Systems provide far more than the opportunity to visualise and share information during each teaching session. The reports produced at the end of each workshop are a precious source of information to analyse and reflect on the effectiveness of the teaching. At the end of each session I download the data generated in the classroom, and I process it to make it available to students. I also use these data to conduct evidence-based research on the pedagogies that I introduced in my module. For each workshop session, responses to assessment questions are coded as ‘correct’ and ‘incorrect’, while responses to self-assessment questions are coded as ‘confident’ and ‘not-confident’. Cross-tabulating the results, I can demonstrate that students who answer to questions correctly (incorrectly) are generally confident (not confident) with their skills. Thus, students generally display (and develop along the year) reasonably good self-assessment skills.   To evaluate the effectiveness of the Peer-Instruction pedagogy, I compute the difference between the proportion of correct responses obtained between the first and the second time each assessment question has been asked, which I call ‘Class Learning-Gain’. The Learning Gain is the measure of effectiveness of Peer-Instruction: the higher the proportion of students who learnt how to reach the right answer by discussing with their peers, the higher the Learning Gain. Thus, using regression analysis, I demonstrate that: (i) learning gains are not dependent on student confidence, and (ii) learning gains are higher when the initial proportion of correct responses is low. Both these results support the view that Peer-Instruction is indeed a powerful pedagogy, which enables both confident and not confident students to learn, and allows the equalising of learning across all students in the classroom. Thus, aside from the ‘buzz’ raised by the advocates of Peer-Instruction in recent times, my approach shows that its success can be validated by evidence.   But what do students think about this? Personally, I do not believe that asking students whether they enjoy being taught in a particular way constitutes a very useful mechanism to evaluate pedagogical effectiveness. Students are likely to enjoy being taught in an interactive way, but this does not necessarily imply that they are learning. Therefore, I decided to proceed through a different approach. In my first lecture I explained to the students that they would be taught through Peer-Instruction and I asked them to share their views on the statement that: ‘students teaching to students can be even more effective than lecturers teaching to students’. At the end of each workshop, I would then ask students to rate the statement that ‘they learnt more Economics by discussing material with each other’. Contrasting the two sets of evidence led to a clear result: while students’ initial opinion of Peer-Instruction was not very high, the great majority of students found it beneficial when assessing their learning experience at the end of each workshop session. This result was also re-enforced by an informal end-of-module survey, where I asked students to identify the component of the module (within lectures, seminars, workshops, office hours, and VLE learning), which had the strongest impact on their learning. I particularly like this approach, because it is not based on leading questions, and it is completely de-contextualised from the discussion about Peer-Instruction. Questionnaire results show that more than 50% of the students thought that workshops (where Peer-Instruction took place) were the most important component of the module, with other preferences distributed almost uniformly across seminars, lectures, and other components. Asking students about what makes them learn is certainly more useful than asking them whether they enjoyed what they did, and I would argue that taking a step aside from the big debate on student satisfaction displays the potential to uncover much more useful insights about our learning and teaching practice.  Ethical considerations  Since I described a teaching evaluation method that makes intense use of student data, I think it is opportune to spend some time discussing the ethical implications for research. The advent of learning analytics has raised serious concerns about the use of student information, ways to obtain consent, and data sharing procedures. All my research has undergone a rigorous ethical scrutiny prior to being conducted. Filing an application for ethical approval might be looking as a daunting task, but I have some suggestions to those who want to embark on producing evidence-based research on learning and teaching data.  First of all, I would claim that if the research that you are conducting is related to teaching activities that would take place irrespectively from your research agenda, so ethical concerns should be already less worrying. This is not research whereby you would alter anybody’s learning and teaching experience with the purpose of writing a paper; it is research based on regular teaching practice, which would occur nevertheless. For the same reason, it should be easier to obtain approval for opt-out procedures (whereby students take action if they want to leave the study) rather than opt-in, which requires lengthy filling of forms: an impractical solution when dealing with large student cohorts.   Additionally, the data collected allows for easy and quick anonymisation of responses, which can also be performed by a third party once records have been matched longitudinally. If you are both the lecturer and the researcher in a project similar to mine, there are both advantages and disadvantages. In order to avoid dependant-relationship issues, any communication about the project, and students’ intentions to opt-out, will need to be handled by a third party who will manage the full dataset. However, as teacher-researcher, you probably already have access to data that can be used to inform your teaching. Ethical implications will arise if you want to use it for research to be disseminated in the public domain, but the step is shorter. As a final remark, I would always recommend that your ethical application highlight the benefits, aside from the costs, that your research projects will bring to the student experience. They are equally important in stating the validity of a research proposal.   Reflections and conclusions  Peer-Instruction appears to be an effective way to promote active learning in large class environments, which is validated by empirical evidence in my own experience. The addition of a self-assessment component to the Peer-Instruction algorithm facilitates the development of self-assessment skills and helps students to focus on their learning tasks. While my teaching methodology was developed within a first-year Economics module, I would envisage that this approach can be seamlessly scaled to other Social Sciences and Natural and Mathematical Sciences. Colleagues teaching and supporting teaching in the Humanities can also benefit from adopting this pedagogy, as responses to assessment questions do not necessarily need to be ‘right’ or ‘wrong’. Examples could be applications to ethical or legal dilemmas (such as those considered in Medical Sciences, or Philosophy, or Law), where information is gradually released to the students in order to lead discussion about controversial judgement-valued statements.   Whichever the context, the fundamental challenge is not the actual implementation of the pedagogy, but the design of questions that can support students in problematicising learning. In other words, I would argue that - aside from testing notions - multiple-choice quizzes combined with Peer-Instruction can generate deep learning, if the questions are carefully crafted. Independently from any research agenda, it is also important: (i) to use the data generated in each session to reflect on our teaching and improve following sessions, as well as (ii) to share reports with the students, so that they can also reflect on their performance, and develop independence and self-regulatory behaviours.   The use of technology can, of course, make our job much easier. But, again, we ought to remember that good pedagogical design is much more important than adopting the most recent technological innovation for its own sake. Training on the use of the technology should be always combined with training on how to devise good pedagogical design. I shall welcome further enquires on the features of this pedagogical approach, and I am always eager to help and support colleagues interested in conducting evidence-based research on its effectiveness.  Reference: Mazur, E (1997) Peer Instruction: A User's Manual. Prentice Hall, NJ.    Fabio Aricò is a Senior Lecturer in Macroeconomics at the University of East Anglia F.Arico@uea.ac.uk 

Promoting active learning through peer-instruction and self-assessment: A toolkit to design, support and evaluate teaching

https://ueaeprints.uea.ac.uk/58518/1/Arico_2016_Educational_Developments.pdf

Promoting Active Learning Through Peer-Instruction and Self-Assessment: A Toolkit to Design, Support and Evaluate Teaching

Abstract

Similar works

Full text

Available Versions

University of East Anglia digital repository

University of East Anglia digital repository