On campus it is a common site to see the

student population plugged into their life support

machines: the iPod and the phone. From newspapers to

radio, the media are recognising need to embrace the

iPod generation to deliver content, and as Rupert

Murdoch has highlighted, newspapers are in risk of

losing out to the digital world. Should ink and paper

continue to be the media of choice for our students?

What can we do with audio? Is audio feedback the future

to support the learning of the iPod generation? This

paper compares the summative assessment2 results for a

cohort using recorded audio feedback in formative and

summative assignments to that of a cohort who received

formative and summative feedback in an aural and/or

succinctly, written form. The paper presents students’

reflections on the use of audio formative and summative

assessment feedback for a module and considers whether

this type of feedback had a pivotal role in the assessment

process and a significant impact on their academic

performance. The paper proposes a strategy for the

integration of digital audio into assessment feedback to

promote feed-forward student learning

Middleton, Andrew

Nortcliffe, Anne

Sheffield Hallam University Research Archive

Audio feedback for the iPod generationNORTCLIFFE, Anne <http://orcid.org/0000-0001-6972-6051> and MIDDLETON, AndrewAvailable from Sheffield Hallam University Research Archive (SHURA) at:http://shura.shu.ac.uk/14434/This document is the author deposited version.  You are advised to consult the publisher's version if you wish to cite from it.Published versionNORTCLIFFE, Anne and MIDDLETON, Andrew (2007). Audio feedback for the iPod generation. In: International Conference on Engineering Education, University of Coimbra, Portugal, 3-7 September 2007. 3-7. Repository use policyCopyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Users may download and/or print one copy of any article(s) in SHURA to facilitate their private study or for non-commercial research. You may not engage in further distribution of the material or use it for any profit-making activities or any commercial gain.Sheffield Hallam University Research Archivehttp://shura.shu.ac.ukCoimbra, Portugal September 3 – 7, 2007 International Conference on Engineering Education – ICEE 2007 Audio Feedback for the iPod Generation  Anne Nortcliffe Faculty of Arts, Computing, Science and Engineering, Sheffield Hallam University, Sheffield, S1 1WB, UK  a.nortcliffe@shu.ac.uk  Andrew Middleton1                                                             1 Andrew Middleton, Learning and Teaching Institute, Sheffield Hallam University, Sheffield, S1 1WB, UK,  A.J.Middleton@shu.ac.uk 2 In the UK the word 'assessment' is similar in meaning to the terms 'testing' or 'evaluation' used elsewhere in the world. Abstract - On campus it is a common site to see the student population plugged into their life support machines: the iPod and the phone.  From newspapers to radio, the media are recognising need to embrace the iPod generation to deliver content, and as Rupert Murdoch has highlighted, newspapers are in risk of losing out to the digital world.  Should ink and paper continue to be the media of choice for our students? What can we do with audio? Is audio feedback the future to support the learning of the iPod generation?   This paper compares the summative assessment2 results for a cohort using recorded audio feedback in formative and summative assignments to that of a cohort who received formative and summative feedback in an aural and/or succinctly, written form.    The paper presents students’ reflections on the use of audio formative and summative assessment feedback for a module and considers whether this type of feedback had a pivotal role in the assessment process and a significant impact on their academic performance.  The paper proposes a strategy for the integration of digital audio into assessment feedback to promote feed-forward student learning.  Index Terms - Assessment, Digital Audio Feedback, Formative and Summative Feedback INTRODUCTION Around the world educational practitioners are editing and posting video and/or MP3 recordings of their lectures to their Virtual Leaning Environments (VLEs) or to the Web. In 2006 the University of California, Berkeley made lectures available to the general public by podcasting a number of courses ranging from the Arts to Engineering using the Apple iTunesU service, [1].   At another institution this type of digital resource has been found to be very popular with the student population [2].  Elsewhere some academics are even abandoning the delivery of live lectures and providing podcasts of pre-recorded lectures to their students [3].  However a comparative research study of students who attended lectures in person with students who only had access to podcast lectures showed that, the Web-based students didn’t perform as well academically, were less motivated, and were missing an extra layer of learning derived from lecture attendance, [4]. A recent exercise at Sheffield Hallam University asked various stakeholders about the use of audio lectures notes. The stakeholders included students, academics, student support, librarians, and technical support staff. The responses indicated that audio lecture notes can be an excellent supplement learning resource to a module, [5]. Though the concept of digital audio lecture notes is in its infancy, there are a number of practitioners in this field developing a wealth of good practice knowledge.  This paper considers whether the digital audio revolution offers new and valuable opportunities to the assessment process in education.   In an earlier study of students studying English as second language, when provided with audio feedback on cassette, found the recorded feedback indispensable and fed forward the comments into future work. In the same study academics found they were able to provide greater clarity than possible when working within the constraints of written feedback, [6].    The previous research predates the digital revolution, the advent of MP3 recorders, Virtual Learning Environments (VLEs), and the posting of audio feedback by email. We are now able to offer digital audio feedback with the advantage that the loss of data is minimised (cassettes can be easily lost, damaged or wiped).  Even so, the analogue solution indicated that the students preferred audio feedback to that of written feedback, and that this approach was marginally less time consuming for the academic, [7].  Providing feedback to students has been identified as stressful activity for lecturers, [8]. Better student-tutor communication and the constructive design of feedback however can ease this stress, [9].  Therefore any feedback technique that can reduce academic workload and provide more satisfactory communication with students has the potential to bring benefits not only for students, but for the academic.   Previous experience of analogue recorded feedback, [6], has indicated that audio feedback has the potential to encourage self-reflection and to facilitate student learning through feedback and into feed-forward assessment action.  In the long term this has the potential to increase student academic performance. This paper examines the use of MP3 recordings posted to the VLE to provide assessment feedback to support formative and summative assessment in a portfolio of assessments.   APPROACH The audio feedback was applied to further support a module that successfully adopts student supplementary teaching methodology where-by the class of students design the Coimbra, Portugal September 3 – 7, 2007 International Conference on Engineering Education – ICEE 2007 course, and subsequently, in groups, deliver a ½ hr presentation on a topic from the course content, produce sample phase test questions on a topic and apply all topics in practice to produce and maintain a group product.  Previous research has indicated that introduction of this approach in the right context can have positive outcome on student learning, [10].  The course is further supported by: • Supplementary lecturing by the academic in remaining time in the one hour lecture slot, covering areas not covered by students and answering further student queries on the topic; • Peer and self-assessment; • Timetabled academic supported laboratories; • E-learning facilities (Blackboard), providing e-learning resources, materials discussion groups, group space, file exchange, email, and sample phase test, etc. Previous research has demonstrated that a greater level of student learning is achievable using this resource as it can contribute to the creation of a learning community through the use of its email, file exchange, on-line discussion and group space tools, [12].   As with any student assessment feedback one hopes that audio feedback will promote student self-reflection and promote learning.  In this case it was envisaged that audio feedback would complement the multi-facet level of learning already promoted through: • student supplementary teaching in conjunction with peer and self-assessment, Figure 1, [11-12] • sample phase test question research and development in conjunction with peer and self-assessment, Figure 2, [12] • group product generation and maintenance in conjunction with self and peer assessment, Figure 3, submitted at the end of the semester            FIGURE. 1 COMPLEX CYCLICAL MODEL : STUDENT SUPPLEMENTARY TEACHING              FIGURE. 2 COMPLEX CYCLICAL MODEL: STUDENT PHASE TEST QUESTIONS                   FIGURE. 3 COMPLEX CYCLICAL MODEL: GROUP PRODUCT GENERATION AND MAINTENANCE  In the case of the group product generation and maintenance previous student cohorts have been required to demonstrate the product prior to and after a period of maintenance.  At the initial demonstration the students primarily receive formative feedback on the product aurally and by being provided with ticks/rings marked onto an assessment grid pro forma. An example of section of the assessment pro forma used is illustrated in example in Figure 4.  The provision of assessment criteria assists the student in self-regulating their assessment performance, [14] and combined with assessment discussion in the lab aids clarification of the assessment criteria. In previous years very concise written comments of just a few words were added to a marking sheet highlighting any future product development for the maintenance period. As the primary feedback is delivered aurally to the students, and written comments are brief due to the constraints of the demonstration schedule (on average 7 minutes for each student). This has resulted in students forgetting or misinterpreting the formative feedback later.                    FIGURE. 4 ASSESSMENT CRITERIA FEEDBACK SHEET Teachers: • topic research • question development • question assessment • feedback Students Learn Teachers: • preparation • deliver • assess • feedback Students Learn Teachers: • formative feedback • assessment • summative feedback Students Learn  • product development and application • maintenance • peer and self assessment  ACADEMIC FEEDBACK FORM: ASSESSMENT GUIDE FEATURE ETC PASS (40%-49%) ETC 1 (70%-100%) LEARNING OUTCOMES Under-standing of internet concepts and theory  Demonstrate some understanding of internet theory, e.g. implications of security, types of languages available for web development, etc.  Demonstrate and excellent understanding of internet theory, which language for which process and why, i.e. in terms of graphics, security, and implemented such strategies, i.e. login and passwords. (2) Critique of on-line system  Provided some analysis of the performance of the site with respect to accessing the site and user friendliness, etc.  Extensive evaluation of the website's performance, download times from different platforms, access and type of access, and location.   Detailed conclusion of improvements to the site, during maintenance period demonstrated improvements to the site. (3) Coimbra, Portugal September 3 – 7, 2007 International Conference on Engineering Education – ICEE 2007 This partial retention of the feedback results in a lack of coherence in the extra level of learning, Figure 5. It therefore seemed like a good idea to record the feedback conversations and then to post them to the group space in the Blackboard VLE. Given that the recordings capture, not only the academic's comments, but also the student's acknowledgement and personal constructions from the feedback process in the lab, it was hoped that the application of theory would be promoted and that the 'learning noise' would be reduced. It was anticipated that this process would compliment the original multi-facetted approach envisaged for this assessment, Figure 3.      FIGURE. 5 COMPLEX CYCLICAL MODEL: IN REALITY GROUP PRODUCT GENERATION AND MAINTENANCE RESULTS IN PRACTICE Assessment Results The benefit of audio feedback on assignments can be measured by comparing the 2006-2007 cohort for this module to two previous years. In 2006-2007 ten groups, with an average membership of three students, cohort of 28 students, engaged with the portfolio of assessment process; volunteered, researched and delivered; • a lecture on a module topic • a sample set of phase test questions • a product, and • demonstrated maintenance of the product However four students did not complete more than one component of the assessment process, they either did not engage with the product, sample phase test questions or lecture.  The behaviour of these students was typical of their engagement on all their modules. The 2005-2006 cohort was composed of ten groups with an average membership of 4, cohort of 41 students. Only 9 of these groups completed the portfolio of assessments. In one group one student disengaged with the group who went on to complete all the assessments.  The tenth group failed to demonstrate or submit a product.   The 2004-2005 cohort was made up of ten groups of an average of three student members, cohort of 30 students. This cohort engaged with all the assessments in the portfolio, but four students disengaged from the module and completed no assessments. This, however, was consistent with their profile on other modules on the course. It should be noted that for this cohort of students the students were not required to research and generate sample phase test questions, but instead to post  an e-summary of a module topic on the VLE.   Only the 2006-2007 cohort Blackboard (VLE) group space received postings of summative digital audio feedback for each assessment and, importantly, received audio formative feedback posting on the product development.   This was intended to add to their understanding or refresh their memory with regard to the maintenance actions and improvements to the product and documentation. Previous cohorts, as indicated in the approach, only received aural formative feedback during their product demonstration, though they were provided with some written comments to take away.   Table I shows that the average student grade for the product assessment submission for the 2006-2007 cohort did not improve in comparison to previous years, and was in fact marginally worse.  These results may indicate that short written comments on the feedback may be more effective into prompting the students into action for the final submission rather than listening to audio formative feedback, which requires an opportunity (time and place) for students to make mental or written note for follow up action.  TABLE I:  PRODUCT ASSESSMENT FINAL SUBMISSION STATISTICS COHORT COHORT SIZE AVERAGE STANDARD DEVIATION 2006-2007 28 52 10.1 2005-2006 41 57 12.3 2004-2005 30 55 13.3 TABLE II: LECTURE ASSESSMENT FINAL SUBMISSION STATISTICS COHORT COHORT SIZE AVERAGE STANDARD DEVIATION 2006-2007 28 63 8.0 2005-2006 41 60 7.8 2004-2005 30 55 11.1 TABLE III: SAMPLE PHASE TEST QUESTIONS ASSESSMENT FINAL SUBMISSION STATISTICS COHORT COHORT SIZE AVERAGE STANDARD DEVIATION 2006-2007 28 52 8.6 2005-2006 41 55 11.3 In the case of the other portfolio assessments the audio feedback was summative, therefore would not expect impact on results illustrated in Table II-III.  However the formative feedback is relevant, but somewhat limited to other similar assessments in other, future modules. Analysis of the module results for the other portfolio of assessments, Table II-III indicates that with each cohort students are becoming more proficient at researching and delivering a lecture, but are weaker in interpretation and generation of sample phase test questions. Teachers: • formative feedback • assessment • summative feedback • Students Learn  • product development and application • maintenance • peer and self assessment  

Audio feedback for the iPod generation

Abstract

Similar works

Full text

Available Versions

Sheffield Hallam University Research Archive