There are various software technologies that allow capture and redelivery of lectures. Most of these technologies however rely on the use of proprietary software, often requiring extra efforts from the lecturer in terms of the initial preparation of the lecture material, or in editing and annotating after the lecture to make the material suitable for the students. To review the material students then require access to the proprietary software. This paper describes a system for the lightweight capture of lecture presentations, based on the use of a low-cost large interactive display surface, together with standard Microsoft PowerPoint™ presentation software. The captured version of the presentation includes the original lecture slides, graphical annotations made by the lecturer during the lecture, and the audio recording of the lecture; all saved as a PowerPoint file. In addition, the system adds some annotations and index slides to allow quick and easy access to different segments of the presentation. Presentations can be replayed in part or in full as required, preserving all of the content of the live lecture

Apperley, Mark

Jansen, Samuel Thomas

Jeffries, Amos

Masoodian, Masood

McLeod, Laurie

Paine, Lance

Rogers, Bill

Thomson, Kirsten

Voyle, Tony

English

Name not available

Lecture capture using large interactive display systems

Research Commons@Waikato

Lecture Capture using Large Interactive Display SystemsMark Apperley, Sam Jansen, Amos Jeffries, Masood Masoodian, Laurie McLeod,Lance Paine, Bill Rogers, Kirsten Thomson, Tony VoyleDepartment of Computer ScienceUniversity of WaikatoHamilton, New ZealandM.Masoodian@cs.waikato.ac.nzAbstractThere are various software technologies that allowcapture and redelivery of lectures. Most of thesetechnologies however rely on the use of proprietarysoftware, often requiring extra efforts from the lecturer interms of the initial preparation of the lecture material, orin editing and annotating after the lecture to make thematerial suitable for the students. To review the materialstudents then require access to the proprietary software.This paper describes a system for the lightweight captureof lecture presentations, based on the use of a low-costlarge interactive display surface, together with standardMicrosoft PowerPoint™ presentation software. Thecaptured version of the presentation includes the originallecture slides, graphical annotations made by the lecturerduring the lecture, and the audio recording of the lecture;all saved as a PowerPoint file. In addition, the systemadds some annotations and index slides to allow quickand easy access to different segments of the presentation.Presentations can be replayed in part or in full asrequired, preserving all of the content of the live lecture.1: IntroductionOver the past few years the use of software suchMicrosoft PowerPoint™ has become an integral part oflecture preparation and delivery. Lecturers prepare theirmaterial using this presentation software, and then eitherdeliver their lecture using the presentation software, orprint their notes onto transparencies and use overheadprojectors during the actual delivery.One of the main reasons for the use of printedtransparencies, rather than direct use of the presentationsoftware, is because it is easier to annotate the lecturematerial on transparencies during the course of a lecture.Although most presentation software allows annotation ofslides during presentation, it is more difficult to use acomputer mouse to draw or write on a slide than it is tosimply use a pen to annotate a transparency. Also, in mostpresentation software, such as PowerPoint™, theannotations aren’t added to the slides, which means thatwhen one moves from one slide to the next then all theannotations on the previous slide are lost. Returning to aslide means returning to the slide as it was originallystored, not as it was last seen.A further problem in the use of existing presentationsoftware is the lack of support for redelivery of the lecturematerial. Conventional presentation software at bestallows one to convert the lecture notes to Web pages andput them online for students to use after the lecture.However, this is clearly less than satisfactory because thestudents then only get the notes without the annotations,and more importantly the audio presentation of thelecturer.Although there is software which allows recording oflectures, both in terms of annotations and audio content,the majority of these tools place significant overheads onthe lecturer to develop their material in a suitable form orusing a particular application [5, 4]. These overheadsoften divert the presenter from the real purpose, thedelivery of the content, and focus attention on the form.The effort required is typically far in excess of thepreparation of the same material for live delivery.This paper describes the LLC (Lightweight LectureCapture) software that has been developed to support therecording of lecture presentations for review andredelivery. LLC is based on standard PowerPoint™presentation software and requires no effort in addition tothat for the original live presentation. The originalpresentation needs to be produced in PowerPoint™format, and that standard application is required for thesubsequent viewing of the recorded presentation. Lecturecapture is best achieved through a large interactivedisplay surface technology, while review and redeliveryrequire only a conventional computer display and soundcard.2: Large Interactive Display SystemThe use of large interactive display systems in teachingand group work environments is not new. Systems such asLiveboard [3] and Smartboard [6] have been available forsome time. The main problem with these technologies,however, has been their high cost that has restricted theiruse to mainly business meetings.As part of an ongoing research project at theProceedings of the International Conference on Computers in Education (ICCE’02) 0-7695-1509-6/02 $17.00 © 2002 IEEE University of Waikato we have been working ondeveloping a low-cost Large Interactive Display Surface(LIDS). A typical LIDS system [1] is an officewhiteboard-sized screen functioning as a computerdisplay. Attachment of Mimio™ [7] digitisationequipment at the front allows the users to directly interactwith the screen using a pen, which serves a similarfunctionality to that of a mouse. A standard data projectorand a rear projection screen are used so as to allow singleuser, or a small group of users, to work directly in front ofthe screen without occluding the image. Figure 1 shows aprototype LIDS system in use an office environment.Even the basic LIDS system (which is just a largescreen with a pen used as a mouse) is attractive to use in anumber of settings such as teaching, demonstration,software evaluation and testing. The large display makesit possible for a group of people to sit and watch what isbeing presented to them, or alternatively they can standaround the display and take turns at contributing.In contrast to a conventional computer and dataprojector setup the operation of a LIDS system iscompletely transparent. The person operating the systemcan work entirely in view of their audience. This isparticularly powerful when demonstrating software ordiscussing usability issues. Even for the individual userthe directness of object manipulation is useful. Ratherthan tinkering with a mouse out of view of an audience, orto the side of the display, the user actually touches objectimages with the pen.Furthermore, in group work situations where theshared document presented on the screen is the focus ofeveryone’s attention, everyone can see what is beingdone; creating a new interaction paradigm of “what isdone is what is seen being done.”However, our objective in developing the LIDS systemwas not just to use it with conventional computerapplications that have often been developed for singleusers working behind a typical computer display. Ratherthe aim was to trial it with specific software designed tosupport small group interaction in teaching or businesssettings. One such use of LIDS is for teaching purposes,which will be discussed here.3: Lecture Delivery ModelOur lecture delivery model assumes that lecture notesare prepared before the presentation using standardMicrosoft PowerPoint™ software. The notes can rangefrom a simple set of outlines, perhaps just with majorheadings, through to a complete set of presentation slides.Alternatively, the presenter could start with a single blankslide, and simply use it as an electronic whiteboardsurface; adding further slides when needed.During the presentation the Lightweight LectureCapture (LLC) software, which acts as an overlay toPowerPoint™, allows the presenter to make annotationsand additions to the individual slides using the Mimio™pens. These annotations are written back into the originallecture notes at each slide transition in standardPowerPoint™ format.The presenter’s voice can also be recorded by the LLCsoftware and is written back to the original presentationfile, in a similar fashion to the annotations. To get goodquality recording it is best for the presenter to wear aheadset microphone.Since the annotations, additions, and audio contentsare written back into the PowerPoint™ lecture notes, it ispossible to further review and do basic editing of theslides after the presentation – making changes likeremoving accidental annotations. Some aspects of therecording are more difficult to edit with PowerPointhowever. In particular presenters may wish to workthrough the audio content to improve clarity and removeirrelevant comments, coughs and hesitations. It was notthe original intention to encourage such editing, because itpartly negates the advantage of lightweight capture.However, user testing alerted us to the fact that somepresenters would feel uncomfortable if they were unableto make corrections before publishing their recordedpresentations. In response the LLC system includes autility to allow editing of the audio recordings[2]. It isalso possible that in the future other tools will be added toprovide LLC with more advanced lecture-editingcapabilities.Although lecturers may use tools such as LLC forcreation, recording and editing of their presentations, wesee it as very important that the presentation slides arekept in a standard format so that they can be reviewed byand redelivered to students, without the students needingaccess to specialised software. The LLC system usesPowerPoint files as its storage medium. Therefore anystudent with access to PowerPoint can review aFigure 1: A prototype LIDS system in useProceedings of the International Conference on Computers in Education (ICCE’02) 0-7695-1509-6/02 $17.00 © 2002 IEEE recorded presentation. In addition the LLC systemprovides a utility to translate a recorded presentation intoHTML, complete with audio content, although lackinganimation. The HTML form can be viewed with any webbrowser, and can be made available over the internet froma web server if required.4: Lightweight Lecture Capture SoftwareAs mentioned, during lecture presentations theLightweight Lecture Capture software operates on top ofPowerPoint™, intercepting interactions and taking controlof the presentation package. Once a presentation is underway, LLC intercepts Mimio™ pen activities, recordingand displaying pen strokes for later addition to thepresentation. LLC also recognizes a set of pre-definedgestures; allowing the presentation to be controlleddirectly using the pen, rather than from a separatekeyboard, mouse, or other additional device.When the user indicates they wish to advance to a newslide, LLC takes all of the recorded pen annotationscaptured for the current slide, together with the audiorecord, and integrates these back into the PowerPoint™presentation for that slide. This is an important aspect ofLLC, because it means that presentations can be replayedusing only the standard software.Annotations are crudely yet effectively animated in therecord using PowerPoint™ entry animations, such as“wipe left” or “wipe right”, chosen automatically to bestmimic the actual pen path. Because the annotations arewritten into the presentation at each slide transition, whena slide is revisited during the presentation or afterwards,all of the previous annotations for that slide can be seenand replayed.Figure 2a shows an original slide from a typicalPowerPoint™ presentation, while Figure 2b shows thesame slide after it has been annotated by the presenter (acircle and an arrow have been added to the image).Figure 2b also shows controls and indicators along thebottom of the slide that have been added by the LLCsoftware. These controls (enlarged in Figure 3) include atimeline of the recorded audio track, showing periods ofsound (bars) and periods of silence (gaps between bars).Arrows beneath the sound bars are turned on and offduring replay to show which audio segment is currentlyplaying. In Figure 3 the arrow shows that the secondaudio fragment is active. On replay the audio can beallowed to play automatically in sequence, or individualaudio bars can be clicked to play their part of the soundtrack. The small dashes above the level of the sound bars,visible in Figure 3 to the right hand side are a set ofindicators showing when the slide was annotated. Eachslide also includes a ToC button which links it to anadditional slide inserted by LLC at the start of thepresentation – the Table of Contents slide (Figure 4).A simple gesture recognizer within LLC allows theMimio™ pen to be used to control the presentation.Supported gestures include a right facing arrow toFigure 2: a) the original slide b) annotated slide with LLC controls addeda bAudio recordings AnnotationsTime slider Slides indexFigure 3: The controls added by LLCProceedings of the International Conference on Computers in Education (ICCE’02) 0-7695-1509-6/02 $17.00 © 2002 IEEE advance a slide, a left facing arrow to retreat through thepresentation, and a scribble action to delete an annotationon the current slide. Gestures can also be used to editdifferent parts of the original PowerPoint™ slides duringthe presentation. A crude character recognition system,using the Grafiti character set is included on anexperimental basis. This allows addition of neatlyformatted text.During review or redelivery, which can be carried outon a conventional display screen, the presentation can beindexed at the individual slide level using the standardPowerPoint™ slide sorter view, or the LLC generatedtable of contents as shown in Figure 4.Within an individual slide, animations and audio aresynchronized according to elapsed time, although asearlier explained, individual graphic components are onlycrudely animated in themselves. Provision is made tohave long silences in the audio minimized to one second,and the annotations resynchronized, again using elapsedtime. Further, within a slide, a mouse click will skipforward to the start of the next graphic animation,allowing rapid browsing of both the graphic elements andaudio at the annotation level.5: EvaluationA pilot study of the use of the LLC software along withthe LIDS system has been conducted. In this brief studythe LLC software was used to capture the audio contentsand annotations made during two lectures given to a classof around 20 students. The students were asked tocompare the LIDS-based presentations with standardwhiteboard-based presentations.The majority of the participants found LIDS generallybetter than whiteboards. They were particularly positiveabout the fact that LIDS made it easy to change andannotate presentations to include, for instance, discussionsinitiated by the students.The participants were also asked to compare LIDSwith other presentations tools such as blackboards andoverhead projectors. In this comparison LIDS was foundto be at least as highly effective as the other presentationtools.The students who took part in this study wereencouraged to review the lecture notes online after theirclass presentations. Some of the student who reviewed thelecture presentations did so in order to review an entirelecture. Others just searched through the presentation toreview particular areas they had not fully understood inthe live presentations.The length of time the students spent reviewing thematerial varied from ten minutes to four hours. Manyparticipants felt that the ability to review the lecturepresentations was beneficial. The most frequentlyidentified benefit was the ability to access material missedin class. Other participants found the review ability usefulin reinforcing or improving their understanding of thematerial covered. The flexibility to review materialmultiple times and at the participant’s own pace were alsomentioned. The audio was identified as a useful featuregenerally, and specifically in assisting with theunderstanding of the material reviewed.6: ConclusionsThis paper has described a prototype lightweightlecture capture system called LLC. LLC manages theshowing of presentations prepared using Microsoft’sPowerPoint system. It provides mechanisms forcapturing annotations made to slides during apresentation, as well as the audio content of thepresentation. Most importantly LLC requires little or noeffort additional to that normally made in preparing alecture. Its only requirement is that the presenter usePowerPoint™. In fact a range of presentation styles andlevels of preparation is supported. A lecture can bedelivered and recorded with no advance preparation at all.The presenter simply starts with a blank PowerPointslide and writes or draws as they go, in the same way thatthey might with a conventional whiteboard.Figure 4: Table of contents generated by the LLC softwareProceedings of the International Conference on Computers in Education (ICCE’02) 0-7695-1509-6/02 $17.00 © 2002 IEEE LCC adds all recorded information back into thepresentation PowerPoint file, complete with timing dataand animation settings to allow automatic replay of eachslide, synchronising annotation with sound. As a resultthe LCC software in not required for replay. Students canuse PowerPoint alone to review a lecture. A table ofcontents and timing components in the finished recordingprovide quick access to parts of the presentation when astudent wants to review only part of the material. Reviewand redelivery can also be done using standard webbrowsers if the annotated lecture slides are converted toWeb pages using LLC’s HTML output utility.A preliminary study of the LLC software in real lecturesettings has demonstrated its effectiveness in capturing theannotations and audio contents of presentations. Thisstudy has also shown that the students have found LLCgenerated slides very useful for reviewing the contents ofpresentations after the lectures.Further development of the LLC lecture audio andannotation capture and editing tools is currentlyunderway. Experiments with speech recognition softwareare described in [2].7: AcknowledgementsThe research described in this paper has been carriedout with the support of the Foundation for Research,Science and Technology under the New EconomyResearch Fund contract number UOWX9910.8: References1. Apperley, M, Dahlberg, B., Jeffries, A., Paine, L.,Phillips, M., and Rogers, W. (2001), Applications of LargeInteractive Display Surfaces, Proceedings of SIGCHI NZSymposium on Computer Human Interaction, PalmerstonNorth, New Zealand.2. Apperley, M, Edwards, O, Jansen, S, Masoodian, M,McKoy, S, Rogers, B, Voyle, T, Ware, D. (2002);Application of Imperfect Speech Recognition toNavigation and Editing of Audio Documents, Proceedingsof SIGCHI NZ Symposium on Computer HumanInteraction, Hamilton, New Zealand.3. Elrod, S., Bruce, R., Gold, R, Goldberg, D., Halasz, F.,Janssen, W.; Lee, D., McCall, K., Pedersen, E., Pier, K.,Tang, J., and Welch, B., (1992), Liveboard: A LargeInteractive Display Supporting Group Meetings,Presentations and Remote Collaboration. ConferenceProceedings of ACM SIGCHI 1992, 599-607.4. Gehne, R., and Jesshope, C. (2000), Tools for theProduction of Small-Footprint, Low-Bandwidth,Streaming Multi-Media for Distance Education,Conference Proceedings of Lifelong Learning 2000, 240-244.5. Jesshope, C. (2000), The Use of Multimedia in Internaland Extramural Teaching, Conference Proceedings ofLifelong Learning 2000, 257-262.6. Smart Technologies (2002), http://www.smarttech.com/.7. Virtual Ink Corporation (2002), http://www.mimio.com/.Proceedings of the International Conference on Computers in Education (ICCE’02) 0-7695-1509-6/02 $17.00 © 2002 IEEE 

Application of Imperfect Speech Recognition to Navigation and Editing of Audio Documents,

Lecture capture using large interactive display systems

Abstract

Similar works

Full text

Available Versions

Name not available

Research Commons@Waikato