Renshaw, T

Soosay, MA

English

Leeds Beckett Repository

ALT JOURNAL NUMBER 7: WINTER 2009 25discussionThis paper describes how students on a Level 6 (final-year undergraduate) Human Computer Interaction (HCI) module were given access to an eye tracker with a view to extending their knowledge of the technologies used in HCI for evaluation purposes. Learning to use this device gives students a distinct advantage in the job market as eye tracking is becoming increasingly accepted as a means of assessing users’ experience by professional usability and design experts. However, the benefits were not all one way. The introduction of the eye tracker also brought about a review of pedagogical benefits of this access in terms of the application of learning theory. Changes in the students’ performance and feedback are examined as indicators of enhanced student experience.drivers for changeThe HCI module has been offered as an option to Innovation North students for more than a decade. A strength of the module is that students learn evaluation techniques that are highly applicable to their final-year projects and relevant to a career in HCI. The dynamic nature of the advances in HCI means that the module content has to be regularly updated and two years ago the module review indicated that there was a clear need to enhance the module by incorporating eye tracking. The capabilities of this technology mean that user interaction can be captured unobtrusively in real time, opening up new horizons in the study of user experience evaluation.eye trackingThe University was an early adopter of eye tracking technology to enhance its HCI research capabilities and purchased a Tobii 1750 which has since become one of the more popular systems in evaluation practice, according to the sole UK agent for Tobii eye trackers (Acuity ETS, 2009). The eye tracker calculates and records the position of the eye of a person sitting in front of a workstation screen every 50th of a second from reflections of infrared light emitted by a source at the base of the device. The eye tracker is totally non-invasive and safe.A well-conducted eye tracking session requires detailed planning and results in a large volume of data as, in addition to time-stamped eye gaze locations, the eye tracker records a variety of other user interactions. The evaluator therefore has quite a broad range of data that needs to be analysed. Space does not permit a detailed description of eye tracking and its use as an interface evaluation technique here. For such further information, the reader is referred to Duchowski (2007) and Webb and Renshaw (2008), while Dix, Finlay, Abowd and Beale (2004) provide an extensive and comprehensive view of HCI and usability evaluation techniques.learning theoryWe decided to reflect upon the incorporation of eye tracking technology into the module and to justify the envisaged benefits in the light of Kolb’s experiential theory (1984). We are aware of the many learning theories that exist but feel that Kolb’s experiential theory matches our own experiences, gained over years of teaching practice, of how students seem to learn. Kolb’s model consists of a cycle of four stages which the learner experiences while learning. The four stages can also be represented as opposite ends of two continua of learning preferences: one comprising learning through [concrete] experience (feeling, intuition, emotion) rather than through abstract conceptualisation (thinking, rational, logical), the other being learning through doing (active experimentation) rather than watching (observation, reflection) (Kolb, Boyatzis and Mainemelis, 1999). The cycle can be represented as shown in Figure 1.Enriching student experience through access to novel technologyTony Renshaw and Meg soosayABSTRACT  CONCEPTUALISATIONFigure 1: The Kolb Experiential Learning Cycle (adapted from Brooke, 1995)➦➦ ➦➦CONCRETE EXPERIENCEACTIVE EXPERIMENTATIONREFLECTIVE  OBSERVATIONALT JOURNAL NUMBER 7: WINTER 200926Kolb believed that a learner could enter the cycle at any point and that the cycle could be repeated many times. Individuals vary in their position on the continua (so-called learning styles) and their learning styles can vary throughout that individual’s development. These dynamics should influence the methods of teaching deployed to optimise and align them with an individual’s learning style. It was for these reasons that we decided to incorporate a practical experience of conducting a hands-on evaluation exercise using eye tracking. Enabling the inclusion of a range of teaching methodologies into the design and methods of assessment of the module accommodated a range of learning styles.learning designThe drivers for change pose challenges to the effective delivery of HCI. We began by rewriting the learning outcomes, specifically to clarify our thinking about what we were aiming to achieve, guided by Kolb’s experiential theory. The learning outcomes provided a solid basis for deciding what is taught and how, as well as providing students with details of what the module entails, the module’s expectations of them and how their learning will be assessed (Brown, 2009). Learning outcomes itemised below were comprehensible, appropriate to the student’s current goals and career plans, challenging and engaging, yet achievable and assessable (Baume, 2009):1. Demonstrate a critical understanding of current   techniques of task analysis, design approaches    and user interface implementation within    specific human-computer work environments.2. Critically appraise methods and guidelines for    HCI evaluation, and conduct an evaluation using   the eye tracker and questionnaire.3. Demonstrate knowledge of a wide range of    interfacing techniques and styles.4. Assess the various trends towards more    sophisticated interfaces with regard to     multimedia, Web 2.0, inclusive design, virtual    reality and online communities.The learning activities were delivered through lectures and tutorials. The lectures described HCI theories from psychological, physiological and usability perspectives. They were not didactic but delivered in a manner that encouraged student participation. Comprehensive module material with assessment methods, marking criteria, a module guide, a list of frequently-asked questions and feedback stages were uploaded to the Virtual Learning Environment X-stream for students, facilitating access to the right support material at the appropriate stages of the module. For the first three weeks students engaged in teamwork and attended tutorials on design approaches and evaluation techniques. In that period students also selected pairs of websites for evaluation from a predetermined list as part of their module assessment. The evaluation techniques learned not only required assessment of the efficiency and effectiveness of websites but also encouraged discussion of the qualitative aspects of a user’s experience such as appeal and engagement, assessed through questionnaires and interviews. Students prepared detailed evaluation plans, decided the roles to be played by each team member during the evaluation session, the tasks that their participants would be asked to do during the evaluation and how best to present the data collected.A feedforward session was held to guide students through good critical writing. Sadler (2002) stated that exemplars are key examples of products or processes chosen so as to be typical of designated levels of quality or competence. Because exemplars are more concrete than abstract, they are especially convenient for direct ‘viewing’ by academic colleagues and students. Exemplary HCI assessments from the previous cohort were used to guide students through expectations of how to execute a good-quality literature review. Videos prepared by tutors and made available via X-stream demonstrated ways of conducting usability evaluation using an eye tracker. This medium allowed repetition of learning (Barritt and Alderman, 2004). Students conducted an evaluation using the eye tracker and used the data produced by it to support their arguments as they wrote a reflective report of their post-evaluation experiences.ALT JOURNAL NUMBER 7: WINTER 2009 27student achievement and feedbackThe real assessors of the success of a module are its students. The following describes students’ reactions to the HCI module based upon information collected through module evaluation questionnaires. In 2007-08 and again in 2008-09 the module has proved to be both popular and successful. Scores achieved indicated that over 79% of the students were satisfied or very satisfied with the module overall. Table 1 compares the results achieved in 2007-08, when no eye tracker was used, with those achieved in 2008-09, which incorporated the use of the eye tracker.In 2008-09 22 out of 37 responses received referred very favourably to the eye tracker and three of six comments about module improvement referred to wanting more time to be allocated to its use. Below are a few of the comments gleaned from the module evaluation questionnaire completed at the end of the module. In answer to the question “What I liked about the module”, students responded:“I liked the way the assignment corresponded to real life experience, I liked the research part that had to be done for it and I enjoyed creating the assignment portfolio which I feel gave me confidence in my HCI evaluation skills.”“Used the eye tracker, before this module I never knew anything like this existed and I didn’t know a single way on how to analyse a website.”In answer to the question: “What I would like to see improved about the module”, comments included:“more time to understand the eye tracker, and how it works”.ConclusionThe module has been a success, and based on students’ module evaluation there is evidence of some positive improvements in their learning experience as shown in Table 1. By including the eye tracker in the redesigned module, all elements of the Kolb Experiential Learning Cycle were experienced: a concrete experience was furnished by the students experiencing at first hand what it was like being both a participant in and supervisor of a realistic evaluation process; reflective observation took place when the students wrote a report on their experiences and findings; abstract conceptualisation was achieved through students having to select the sites to be evaluated, plan the evaluation, and envisage the tasks that would evidence the usability or otherwise of the sites; and finally active experimentation was achieved by the students actually executing the planned evaluation, obtaining and analysing the results from which they drew their conclusions. Table 1Metric 2007-08 Results 2008-09 Results Number of  students 166 63 Average  assessment %  56.4% 62.0% % failing  12% 14%  % not submitting 6% 10% Those achieving less than 20% 4 1 Those achieving between 20 and 40% 16 8 % completing the questionnaire 44% 68% ALT JOURNAL NUMBER 7: WINTER 200928ReferencesAcuity ETS (2009) Solutions for Usability & UX. Available at: http://www.acuity-ets.com/Solutionsforusability.htm [Accessed 17 September 2009].Barritt, C., and Alderman Jr., F.L. (2004) Creating a reusable learning objects strategy: Leveraging information and learning in a knowledge economy. San Francisco: Pfeiffer. Baume, D. (2009) Writing and using good learning outcomes. Leeds: Leeds Metropolitan University.Brooke, J. (1995) Training and Development Competence: a practical guide. London: Kogan Page. Brown, S. (2009) Preface. In: Baume, D. Writing and using good learning outcomes. Leeds:  Leeds Metropolitan University.Dix, A., Finlay, F., Abowd, G.D. and Beale, R. (2004) Human Computer Interaction (3rd edn). Harlow: Pearson Education Ltd. Duchowski, A.T. (2007) Eye Tracking Methodology: Theory and Practice (2nd edn). London: Springer. Kolb, D. (1984) Experiential learning. New Jersey: Prentice Hall. Kolb, D.A., Boyatizis, R.E. and Mainemelis, C. (1999) Experiential Learning Theory: Previous Research and New Directions. EBLS Research Library. [Accessed 16 September 2009].Sadler, D. R. (2002) Ah! … so that’s ‘quality’. In: Schwartz, P. & Webb, G. (eds) Assessment: case studies, experience and practice from higher education. London: Kogan Page, 130–136.Webb, N. and Renshaw, J.A. (2008) Eye tracking in HCI. In: Cairns, P. & Cox, A. (eds) Research methods for Human-Computer Interaction. Cambridge: Cambridge University Press. Tony RenshawDirector of Usability North and Lecturer Innovation NorthMeg SoosayTeacher Fellow Technology Enhanced Learning TeamInnovation North and Faculty of Arts & Society 

Ah! … so that’s ‘quality’. In:

Creating a reusable learning objects strategy: Leveraging information and learning in a knowledge economy.

Experiential Learning Theory: Previous Research and New Directions.

Experiential learning.

Eye tracking in HCI. In:

Eye Tracking Methodology: Theory and Practice (2nd edn).

Human Computer Interaction (3rd edn).

Preface. In: Baume, D. Writing and using good learning outcomes.

Solutions for Usability & UX. Available at: http://www.acuity-ets.com/Solutionsforusability.

Training and Development Competence: a practical guide.

Writing and using good learning outcomes.

Enriching student experience through access to novel technology

Enriching student experience through access to novel technology

Abstract

Similar works

Full text

Available Versions

Leeds Beckett Repository