Notes to support tutors in using zapper feedback for undergraduate teaching in large groups

Read, D.

English

EdShare

Enhanced feedback to students using data collected using zappers  Background  Students take 12 ‘in-class tests’ during the first year of their Chemistry degree.  These are written tests, often, but not exclusively, in multiple choice format, which are taken during lectures.  They are marked by academic staff, and returned, with feedback taking the form of a generic discussion during the subsequent lecture.  Because students are used to getting frequent individual feedback from teachers at school, it is desirable to improve the provision of personalised feedback in the first year course.  Ultimately, the aim is for students to be able to self-assess their grasp of concepts in order to be able to direct their own independent learning, but many students need a degree of guidance before they are able to do this themselves.  Using zappers in the process of running in-class tests  Although it would be feasible to simply run the in-class tests as a zapper question session using TurningPoint/Powerpoint, this would put all students under pressure to answer each question in a defined time-frame (e.g. 1 minute).  Our preliminary discussions with students indicated that this would not something they would welcome (“I’m not very good at answering questions under pressure” being a common comment).  As such, it was decided that students would complete the test on paper in the usual allocated time (typically 30 minutes), with a quick polling session using zappers to collect their answers at the end of the session.  The expected benefits of using zappers in this way were:  1) Rapid collection of data and automatic marking by the software 2) Automatic processing of the data by the software to allow in-depth question-by-question analysis by academic staff 3) The generation of mail-merged personalised feedback reports based on response data, with reports being e-mailed to students on the day of the test  Putting this approach into practice: trial  This approach was trialled in a limited form with 2 in-class tests at the end of semester 1.  The aim of these trials was to collect the students’ answers using the zappers, while the academic marked the paper to check the correlation between the written answers and the data collected electronically. The zappers were labelled so a specific handset could be allocated to each student, and students were told which zapper to collect by e-mail (using mail merge).  The students were told where to collect their zapper from (those labelled A1-10 were on the bench on the front row of the lecture theatre, with zappers coded B on the second row and so on).  After the session, a feedback report was generated which gave the students their mark and included the response charts generated by the software for each question, allowing the students to see how everybody got on with each question.  No personalised feedback was given in these trial sessions.   Learning points from the trial:   Students were able to collect their zappers without difficulty, but once they had picked up their handset, most students proceeded to move to another part of the lecture theatre, creating ‘traffic jams’. For future sessions, students would be asked to sit in the row of the lecture theatre from which they had collected their zapper.  After marking the test papers, it was found that there was very little error between the zapper data and the written papers (the error was found to be <0.3%). In future, it will be recommended that staff mark a random sample (e.g. 10% of papers) to check that there are no significant discrepancies.  Written papers will be retained in case of dispute by students or to allow for future checking in case academic integrity is brought into question.  During the collection of answers using zappers at the end of the first trial test, it was found that displaying the charts after students had responded to each question led to discussion.  This held up the process of collecting data, and also compromised the integrity of the test. In the second trial, the charts were not displayed, and the collection process was much faster.  In future sessions, test conditions (i.e. no talking) will be enforced throughout the duration of the written test and the polling session.  It was found that question-by-question response data for each student could easily be extracted from the data collected using the zappers. This showed that the creation of feedback reports using mail merge was viable, and it was decided to trial this in two in-class tests in semester 2.  Semester 2 trial: Creating personalised feedback reports  The aim of this approach was to deliver personalised feedback to each student for each answer that they gave in the in-class test.  This was done in the following way (note that the in-class test and polling session were run as described above, taking into account what we learned from the initial trials):  A) Academics wrote a feedback statement for each optional answer for each multiple choice question.  The feedback statements for incorrect answers gave an indication of why the answer was wrong where this was possible. B) The feedback statements were copied into a look-up table in an Excel spreadsheet. By using the ‘VLOOKUP’ function of Excel, this allowed the appropriate feedback statement to be pulled out of the look-up table for each answer given by each student. C) A template feedback report was created in Word, which incorporated screenshots of the charts showing the whole cohort responses to each question.  By using the mail merge functions in Word, the appropriate feedback statements for each student were included in the report, along with their provisional mark. D) The feedback reports were created using mail merge, and each student’s report was saved as an individual file (this feature does not exist in MS Word and required the use of a software add-in, which was installed without difficulty). E) A template e-mail was created using the mail merge feature in Word, which would be addressed to each student individually, and would carry that student’s feedback report as an attachment (it is not possible to add a different attachment to each e-mail using the standard mail merge feature, so another software add-in called ‘Mail Merge Toolkit’ was used to do this. This was also straightforward to use). F) The reports were e-mailed to students on the same day that they had done the in-class test.  Conclusions  This procedure was used with two in-class tests in semester two, with the academic staff involved choosing to mark all of the written papers in each case.  Once again, very few discrepancies were found between the answers given in writing and those submitted electronically, increasing our confidence that in future it will not be necessary for academic staff to mark every single paper manually.  Students commented that the feedback was very helpful, particularly as it arrived in such a timely fashion.  This was especially important in the case of the first of the two semester 2 tests which took place on the last day of term before the Easter holiday. By sending the students a report that day, they were able to do work over the holiday to address any issues arising.  Creating the spreadsheet incorporating the look-up table was fairly time-consuming, although it was found to be a quicker process second time around.  Creating the feedback report template also took some time, notably the capture and inclusion of the charts showing the whole cohort responses to each question, but the value that students place on seeing these warrants the effort.  In conclusion, we believe that this is a valuable addition to our armoury in the delivery of feedback, and we feel it is desirable to extend the use of this approach in the first year, and perhaps the early part of the second year where some students fall behind quickly.  The approach could be used with informal zapper question sessions as well as more formal in-class tests if it is deemed appropriate. 

Enhanced feedback using zappers

Abstract

Similar works

Full text

Available Versions

EdShare