When learners are asked to verbalise their thoughts about multiple mathematical representations, some researchers are left to analyse utterances based on video records of activity which may have ambiguous signifiers. They are also faced with post hoc analysis of paper-based worksheets, in which temporal order has to be guessed. In this paper, attempts to minimise such methodological problems by means of recent technologies such as eye-tracking, tablet PC screen capture, digital video cameras and the latest video analysis tools are illustrated in the context of a study of the effect of varying representational instantiations on learners' problem-solving strategies

Aczel, James

Hodgson, Barbara

San Diego, Jonathan

Scanlon, Eileen

English

Open Research Online (The Open University)

Open Research OnlineThe Open University’s repository of research publicationsand other research outputs’There’s more than meets the eye’: analysing verbalprotocols, gazes and sketches on external mathematicalrepresentationsConference or Workshop ItemHow to cite:San Diego, Jonathan; Aczel, James; Hodgson, Barbara and Scanlon, Eileen (2006). ’There’s more than meetsthe eye’: analysing verbal protocols, gazes and sketches on external mathematical representations. In: 30th annualconference of the International Group for the Psychology of Mathematics Education (PME 30), 16-21 Jul 2006,Prague, Czech Republic.For guidance on citations see FAQs.c© [not recorded]Version: [not recorded]Link(s) to article on publisher’s website:http://class.pedf.cuni.cz/PME30/Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.uk2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-17  “THERE’S MORE THAN MEETS THE EYE”: ANALYSING VERBAL PROTOCOLS, GAZES AND SKETCHES ON EXTERNAL MATHEMATICAL REPRESENTATIONS Jonathan P. San Diego, James Aczel, Barbara Hodgson and Eileen Scanlon The Open University, United Kingdom  When learners are asked to verbalise their thoughts about multiple mathematical representations, some researchers are left to analyse utterances based on video records of activity which may have ambiguous signifiers. They are also faced with post hoc analysis of paper-based worksheets, in which temporal order has to be guessed. In this paper, attempts to minimise such methodological problems by means of recent technologies such as eye-tracking, tablet PC screen capture, digital video cameras and the latest video analysis tools are illustrated in the context of a study of the effect of varying representational instantiations on learners’ problem-solving strategies.  INTRODUCTION When researchers study learners’ sense-making of mathematical representations, verbal utterances are often not enough in isolation. For example, Ainley, Bills and Wilson (2004) recorded students’ computer screen activity: Student: You need times ‘cause you need to that (points to 15 minutes) times twenty. (p. 6) Identifying exactly what the student is signifying by “that” is important for the analysis. Instances such as this occurred many times in the data that Ainley and her colleagues presented. In principle, were researchers to recognize ambiguous signifiers at the time, they could clarify ambiguity by asking participants what they meant. This can violate one of Ericsson and Simon’s (1984) suggested guidelines – researchers should only intervene when participants stop talking and can only say “please keep on talking.” – and potentially break concentration. There are consequently numerous empirical studies using videos of learners’ gestures such as pointing. However, participants do not always point to what they what they are referring. It is also typically difficult from these videos to pinpoint where learners are making inferences: numerical information for example might be extracted from displayed coordinates, from a graph, or from an algebraic expression. Another example of where verbal utterances are not necessarily enough is when learners may prefer to think on paper (Villarreal, 2000). There are also mathematical 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-18  terms that students may not express verbally, and writing or sketching may help them to describe what they want to say. However, without the opportunity to do this in real time, participants may not be able to provide crucial insights into their thinking. Pirie (1996) suggests complementing a think-aloud protocol with paper-based worksheets. However, when writing is analysed after the event, the temporal order of writing and the role of scratch work have to be guessed. In previous work we have attempted to address the challenge by videoing learners writing processes using a camcorder. Due to the limitation of visual range and learners’ movement, it was difficult to see the totality of the process.  Using the latest technologies, a technique has been developed to capture, coordinate and analyse learners’ gazes, real-time writing, utterances and actions. The paper illustrates analytical advantages of using this technique. COMBINED THINK-ALOUD, EYE-TRACKING AND HAND-WRITING PROTOCOLS Previous work analysing video data of students problem-solving with multiple mathematical representations emphasises the need to identify which representation is being considered by a learner as utterances are made and to examine more closely students’ movement between representations (San Diego, Aczel, & Hodgson, 2004). At the same time, there is a need to somehow see what learners record on paper in real-time as it may provide important evidence such as erasure of previous inferences made. Which representations learners pay attention to and what they do with them is crucial in investigating learners’ sense-making of multiple representations. It is important to see how participants are organising the processes of what they do with each particular representation as they search for a solution to a problem. Video of an individual can be combined with other event streams such as screen activity. Doing this using previous technology, however, posed technical, practical, methodological and ethical challenges (see e.g. (Hall, 2000; Powell, Francisco, & Maher, 2003; Roschelle, 2000; San Diego et al., 2004)). Recent technologies offer new opportunities that may address some of the methodological challenges presented in the previous section. To gain insight into what a person is looking at is possible using an eye-tracking device (Hansen et al., 2001). This can also be coupled with real-time writing using tablet PC screen capture. Figure 1 provides an example of what can be collected using these technologies. The two figures on the right are both ‘screen activity’. The upper-right is what the observer sees during the study. The lower-right figure is an image generated by the analysis software where the lines, called ‘saccades’, indicate the path that the eyes took across the screen; and the circles, called ‘fixations’, show where the eye dwelled on an element of the screen for a length of time above a specified threshold. By superimposing fixations and saccades on the interface, the researcher can clearly see shifts in attention. 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-19   Figure 1: Examples of action, writing, screen and gaze videos SAMPLE EPISODE To illustrate, we briefly discuss a sample episode from a lab-based study involving participants with A-level mathematics qualifications or higher.  The study looked at the effect of varying representational instantiations (static images, on learners’ problem-solving strategies. These instantiations were “static” (non-moving, non-changing, non-interactive), “dynamic” (capable of animation through alphanumeric inputs), and “interactive” (directly manipulable graphs). One of the tasks undertaken by participants was “An original cubic function f(x) is rotated 180 degrees about a point (a, 0). What can you infer about the solutions of the new function?” Figures 2, 3 and 4 show snapshot data of a participant working on this task involving multiple representations. 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-20  Figure 2:Talk by the participant  Figure 3: Screenshots of functions considered by the participant  Figure 4: Final record of the participant’s writing (two pages) Start time Action Annotation 00:00:07:10 Talks “You've got your two point not…” 00:00:12:10 Talks “…and the one at two and a half” 00:00:37:20 Talks “Ah ok! I see what the rotation means now.” 00:00:41:21 Talks “So you gonna to get one there... one there” 00:03:02:20 Talks I'm just thinking kind of... how they move... and they are moving in relation to that point there... so it has something to do with the distance from... the point to a… 00:03:20:17 Talks It's the same, obviously. 00:03:24:15 Talks So kind of a way of thinking about that but it is not that (erased -2a) 00:03:45:06 Talks The distance from plus to here is 4.7 00:03:52:10 Talks You kind of have the distance between x and a 00:05:53:06 Talks Let's see if it works 00:06:14:12 Talks Hmmm! Let's go to the next one 00:06:57:08 Talks 7 and 12. WHOA!!! I can do maths! (Laughs) 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-21  ILLUSTRATIVE ANALYSIS OF UTTERANCES, GAZES AND SKETCHES In isolation, the snapshot data does not suggest a strong narrative about the episode. Clearly though, a synchronous replay of talk, screen, writing and gaze data provides a richer basis for analysis than snapshots.   So, for example, the talk “you’ve got your two point not…and the one at two and a half” (00:00:07:10) is ambiguous: “point” can refer to either a graphical point or a point in a decimal number. Thus, it could be interpreted as “2.0” However, following the gaze video (Figure 5), it is clear that the participant is stating that there are two points (i.e. solution points); one is being “zero” and the other one with a value of “2.5”  An expression like “Ah!” is normally an indication of excitement. However, it is difficult to tell what particular event elicited this behaviour. The gazes (Figure 6) show that the participant looked at all numerical representations and corresponding graphic representations during the utterance “Ah Ok! I see what the rotation means now… So you gonna get one there and one there….” (00:00:37:20). It is clear that the participant ignored the equation and only related the numeric with the graphic representations.  Referring to what the participant has written on paper (Figure 7) without seeing it replayed over time, it cannot be seen that the participant made sense of the numbers by assuming that one of the solutions (zero) is twice the point of rotation (-1.5). The participant also assumed that the solution is being doubled and may have been rounded. The participant only uttered “zero goes to -3 and 2.5 goes to -5.5.” but did not utter “-2a” More to this is that in the final paper. “-2a” Figure 5: Gaze video Figure 6: Gaze video Figure 7: Writing video (start of first page) 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-22  was erased.   In Figure 8, the participant then works with the numbers and is trying to connect this to the graph. The participant looked at the numbers (middle figure) and wrote them down (left figure) then she looked at the graphs (right figure) and said, “I’m just thinking how it moves…in relation to that point… it has something to do with the distance from that point… It’s the same obviously… it is not that (erased -2a)… the distance is 4.7…” (00:03:02:20). The sequence of gazes can provide insight into where difficulties in linking representations might be occurring.  Figure 8 Writing (First page continuation) and gaze videos Interestingly, by seeing how the writing proceeds over time, it is possible to tell how this participant was trying to make an algebraic rule (Figure 9). First she performed some numerical calculation: “2-0.73 = 1.27” then “2+1.27=3.3.”. Then, she replaced each number with symbols starting with “2+x” then “2” was overwritten with “a.”. Then she wrote “2 + (a-x)” then again overwrote “2” with “a” and came up with “a + a-x”.           Figure 9 Final first page (writing video)  2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-23  Next, the algebraic rule created was tested using the software to verify if the rule is correct. She said, “let’s see if it works” (00:05:53:06) then copied the numbers and applied the rule (left hand, Figure 10). She clicks on the next figure that reveals the solutions of the new solutions (upper-right figure) and said “Hmmm! Let’s go to the next one.” (00:06:14:12) She tried another set of calculations then again click the next figure (lower-right figure) to verify if her rule is correct and said, “7 and 12! Whoa! I can do maths!” (00:06:57:08)  Figure 10 Writing (final second page) and gaze videos CONCLUSION The short excerpt we have shown here is an attempt to show how the technique of combining gazes, utterances, actions, and writings can be used to investigate learners’ interaction with multiple representations in a computer environment. However, care should be taken in organising the rich amount of data that can be derived (Powell et al., 2003).  It should be noted that eye-tracking evidence needs to be treated with care, because essential information might be picked up by peripheral vision (Hansen, 1991). Nevertheless, in the particular study mentioned here, think-aloud, eye-tracking and writing-capture protocols have started to provide hints about how varying representational instantiations influence learners’ problem-solving strategies. There was evidence, for example, that particular learner described here used both graphic and algebraic strategies when the task was presented in static form, but concentrated on working with graphic representations when a similar task was presented in interactive form. 2006. In Novotná, J., Moraová, H., Krátká, M. and Stehlíková, N. (eds.). Proceedings 30th Conference of the International Group for the Psychology of Mathematics Education, Vol. 5, pp. 17-24. Prague, PME.          5-24  Our view is that these techniques offer potentially interesting developments in understanding students’ engagement with mathematical representations. ACKNOWLEDGMENTS The authors wish to thank: Pascal Mangold of Mangold Software & Consulting GbHM, Microsoft Research in Cambridge through Dr. Fabien Petitcolas for sponsoring a tablet PC; and, from The Open University, Prof. John Mason, Dr. Ann Jones, Will Woods, Phil Downs, Dave Perry, Ekaterini Tzanidou and Dr. Marian Petre for their help and advice. REFERENCES Ainley, J., Bills, L., & Wilson, K. (2004). Constructing meanings and utilities within algebraic tasks. Paper presented at the PME 28: Inclusion and Diversity, Bergen College University, Norway. Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: verbal reports as data. London: The MIT press. Hall, R. (2000). Video recording as theory. In A. Kelley & R. Lesh (Eds.), Handbook of research design in Mathematics and Science education (pp. 647-664). London: Lawrence Erlbaum. Pirie, S. (1996). What are data? An exploration of the use of video recording as a data gathering tool in the mathematics classroom. Paper presented at the Sixteenth Annual Meeting of the International Group for the Psychology of Mathematics Education, North America, Florida State University, Panama City. Powell, A., Francisco, J., & Maher, C. (2003). An analytical model for studying the development of learners' mathematical ideas and reasoning using videotape data. Journal of Mathematical Behavior, 22, 405-435. Roschelle, J. (2000). Choosing and using video equipment for data collection. In A. Kelley & R. Lesh (Eds.), Handbook of research design in Mathematics and Science education (pp. 709-729). London: Lawrence Erlbaum Associates, Publishers. San Diego, J. P., Aczel, J. C., & Hodgson, B. (2004). The effects of technology on making conjectures. Paper presented at the PME 28: Inclusion and Diversity, Bergen College University, Norway. Villarreal, M. (2000). Mathematical thinking and intellectual technologies: the visual and the algebraic. For the learning of mathematics, 20 (2), 2-7.  

'There's more than meets the eye': analysing verbal protocols, gazes and sketches on external mathematical representations

http://oro.open.ac.uk/7173/1/San_Diego_et_al_%282006%29_-_PME_30.pdf

'There's more than meets the eye': analysing verbal protocols, gazes and sketches on external mathematical representations

Abstract

Similar works

Full text

Available Versions

Open Research Online (The Open University)