We describe a graph-based modelling approach to exploring interactions associated with a change in students' affective state when they are working with an exploratory learning environment (ELE). Student-system interactions data collected during a user study was modelled, visualized and queried as a graph. Our findings provide new insights into how students are interacting with the ELE and the effects of the system's interventions on students' affective states

Grawemeyer, Beate

Gutierrez-Santos, Sergio

Holmes, W.

Mavrikis, M.

Poulovassilis, Alexandra

Wollenschlaeger, Alex

Birkbeck Institutional Research Online

Using Graph-based Modelling to explore changes instudents’ affective states during exploratory learning tasksBeate GrawemeyerBirkbeck, University of Londonbeate@dcs.bbk.ac.ukAlex WollenschlaegerBirkbeck, University of Londonawolle01@dcs.bbk.ac.ukSergio Gutierrez-SantosBirkbeck, University of Londonsergut@dcs.bbk.ac.ukWayne HolmesThe Open University, UKwayne.holmes@open.ac.ukManolis MavrikisUCL Institute of Educationm.mavrikis@ucl.ac.ukAlexandra PoulovassilisBirkbeck, University of Londonap@dcs.bbk.ac.ukABSTRACTWe describe a graph-based modelling approach to explor-ing interactions associated with a change in students’ affec-tive state when they are working with an exploratory learn-ing environment (ELE). Student-system interactions datacollected during a user study was modelled, visualized andqueried as a graph. Our findings provide new insights intohow students are interacting with the ELE and the effectsof the system’s interventions on students’ affective states.1. INTRODUCTIONMuch recent research has focussed on Exploratory Learn-ing Environments (ELEs) which encourage students’ open-ended interaction with a knowledge domain, combined withintelligent components that aim to provide pedagogical sup-port to ensure students’ productive interaction. The aim ofthis feedback is to balance students’ freedom to explore alter-native task solution approaches while at the same time pro-viding sufficient support to ensure that the intended learn-ing goals are being achieved [6]. Here we report on recentwork into identifying interaction events that are associatedwith a change in students’ affective state as they interactwith an affect-aware ELE called Fractions Lab. We adopta graph-based approach to modelling, querying and visual-izing the student-system interactions data, extending pre-liminary work in this area reported in [8]. In our graphs,nodes represent occurrences of key indicators that are de-tected, inferred or generated by the ELE, and edges be-tween such nodes represent the “next event” relationship. Incontrast, recent work on interaction networks and hint gen-eration (e.g. [4]) uses graphs whose nodes represent stateswithin a problem-solving space and edges represent students’actions in transitioning between states. That work uses thegraph-modelled data to automatically generate feedback forthe student, whereas we use a graph-based modelling ap-proach to investigate the effects of the system’s interventionsin order to better understand how students interact with theELE with the aim of improving its support for students.2. THE ELE AND USER STUDYFractions Lab is an ELE that is part of the iTalk2Learnlearning platform targeted at children aged 8-12 years whoare learning about fractions. As students interact with Frac-tions Lab they are asked to talk aloud about their reasoningprocess. This speech, together with their interactions, areused to detect students’ affective states using a combinationof Bayesian and rule-based reasoning [5]. Adaptive supportis provided based on the student’s performance and detectedaffective state. The affective states detected by FractionsLab can be ranked according to their effect on learning,based on previous studies (e.g. [7, 3, 1]). For example, beingin flow is a positive affective state as it indicates that thestudent is engaging with the learning task well. Confusion ismostly associated with realising misconceptions, which alsocontributes towards learning, while frustration and boredomare likely to have a negative effect on learning.We conducted a user study in which iTalk2learn was usedby students in a classroom setting. 41 students aged 8-10took part, with parental consent, recruited from two schoolsin the UK. Students were given a short introduction to thesystem. They then engaged with the Fractions Lab ELE for40 minutes. They then completed an online questionnairethat assessed their knowledge of fractions (the post-test).The iTalk2Learn platform logged every student-system in-teraction, such as fractions being created or changed by stu-dents, buttons being clicked, feedback being provided bythe system, feedback being viewed by students, and thesystem’s detection of students’ affective states. This datawas then remodelled into a graph form, according to thegraph data model shown in Figure 1. We see that the datamodel comprises two node types: Event nodes, that cap-ture occurrences of key interactions, and EventType nodes,that hold additional metadata about each event. Edges la-belled NEXT link together successive Event nodes, allowingus to build up a sequence of events that describe the his-tory of student-system interactions as a student works on atask during a session. An edge labelled OCCURRENCE OFlinks each Event node to an EventType node.The data logged by iTalk2Learn was exported as text, parsedand pre-processed using Python and the Pandas and py2neoFigure 1: Graph data model for student-system in-teraction data.libraries, and then loaded into the Neo4j graph database. Toview the resulting data graph we developed a custom visu-alization tool in JavaScript using the Node.js library. Ourtool allows viewing of large-scale changes in affective stateas well as details of event sequences. Having interacted withthese visualizations, we were interested to explore furtherthe kinds of events that contribute towards changes in stu-dents’ affective state as they work with Fractions Lab. Todo this, we used Neo4j’s graph query language, Cypher, toextract the metadata relating to pairs of consecutive eventsthat exhibit a change in a student’s affective state. Thequery below was used to find adjacent Event nodes con-nected by NEXT, and the EventType nodes they are con-nected to by OCCURRENCE OF, such that the affectivestates associated with the EventType nodes are not equal:MATCH (start_event: Event)-[:OCCURRENCE_OF]->(start_type: EventType),(end_event: Event)-[:OCCURRENCE_OF]->(end_type: EventType),p = (start_event)-[:NEXT]->(end_event)WHERE start_type.affective_state in["flow", "boredom", "confusion", "frustration"]AND end_type.affective_state in["flow", "boredom", "confusion", "frustration"]AND NOT start_type.affective_state = end_type.affective_stateRETURN *3. RESULTS AND CONCLUSIONSWe were interested to explore differences in students’ af-fective states and interactions compared with their perfor-mance. Students’ performance, based on the post-test score,was on average 3.83 (SD=1.46; min=0; max=6). A mediansplit of students’ scores resulted in a higher- and a lower-performing group (high: 27 students; low: 14 students). Inorder to investigate which interactions moved students intoa different affective state we used association rule learn-ing (c.f. [2]) over the data returned by the above Cypherquery. We found that students are likely to move from flowto frustration when provided with reflective prompts in thelow-performing group and with open-ended problem solvingsupport in the high-performing group. This might implythat these types of support are imposing too high a cog-nitive demand on students. Additionally, certain interac-tions with their fractions may move both categories of stu-dent from flow to frustration. Viewing high-interruption orlow-interruption feedback may move low or high performingstudents, respectively, from flow to confusion. Finally, weobserved a positive effect of Affect Boost messages for bothcategories of student.These findings extend earlier ones reported in [5] with afiner-grained analysis of students’ affective state changes,identifying several situations where the system’s supportmay need to be modified: (i) reviewing the content of boththe high- and the low-interruption messages, to see if theincidences of confusion can be reduced; (ii) considering ex-tending the provision of reflective prompts and open-endedsupport with additional affect boost messages and hints thatstudents might also select to view, to mitigate against frus-tration; (iii) considering providing more scaffolds when stu-dents are manipulating their fractions, for example addi-tional low-interruption feedback. Exploratory learning en-vironments such as Fractions Lab can generate large volumesof student-system interactions data, making their interpre-tation a challenging task. We have seen here how modellingsuch data as a graph can open up new data visualization,querying and analysis opportunities, leading to new insightsinto how students are interacting with the ELE and the ef-fects of the system’s interventions, with the ultimate goal ofdesigning improved support for students.4. REFERENCES[1] R. S. J. d. Baker et al. Better to be frustrated thanbored: The incidence, persistence, and impact oflearners’ cognitive-affective states during interactionswith three different computer-based learningenvironments. Int. J. Hum.-Comput. Stud., 68, 2010.[2] D. L. Bazaldua, R. S. J. de Baker, and M. O. S. Pedro.Comparing expert and metric-based assessments ofassociation rule interestingness. In EDM, 2014.[3] S. K. D’Mello et al. Confusion can be beneficial forlearning. Learning & Instruction, 29(1):153–170, 2014.[4] M. Eagle, D. Hicks, B. Peddycord III, and T. Barnes.Exploring networks of problem-solving interactions.LAK, pages 21–30, 2015.[5] B. Grawemeyer et al. Affective learning: Improvingengagement and enhancing learning with affect-awarefeedback. User Modeling and User-Adapted Interaction- Special Issue on Impact of Learner Modeling, 2017.[6] S. Gutierrez-Santos, M. Mavrikis, and G. D. Magoulas.A Separation of Concerns for Engineering IntelligentSupport for Exploratory Learning Environments. J.Research and Practice in Inf. Tech., 44:347–360, 2013.[7] R. Pekrun. The control-value theory of achievementemotions: Assumptions, corollaries, and implicationsfor educational research and practice. J. Edu. Psych.Rev., pages 315–341, 2006.[8] A. Poulovassilis, S. Gutierrez-Santos, and M. Mavrikis.Graph-based modelling of students’ interaction datafrom exploratory learning environments. In Proceedingsof G-EDM, at EDM, 2015.

Using graph-based modelling to explore changes in students’ affective states during exploratory learning tasks

Abstract

Similar works

Full text

Available Versions

Birkbeck Institutional Research Online