This study reports the use of automated tutoring and scaffolding implemented in the
module “arithmetic word problem” in the educational technology software
MathemaTIC in grade 3 (age 8 to 10). We examined 246 students with access to
MathemaTIC and receiving tutoring and scaffolding through a one-to-one learning
setting with this technology. The control group (n=226) had access to the same
learning tasks and worked with paper-and-pencil without MathemaTIC but with their
teachers. Results showed that the experimental group finished with higher outcome
scores than the control group. This paper will outline the study and attempts to explain
these results

Haas, Ben

Kreis, Yves

Lavicza, Zsolt

peer reviewedThis study reports the use of automated tutoring and scaffolding implemented in the
module “arithmetic word problem” in the educational technology software
MathemaTIC in grade 3 (age 8 to 10). We examined 246 students with access to
MathemaTIC and receiving tutoring and scaffolding through a one-to-one learning
setting with this technology. The control group (n=226) had access to the same
learning tasks and worked with paper-and-pencil without MathemaTIC but with their
teachers. Results showed that the experimental group finished with higher outcome
scores than the control group. This paper will outline the study and attempts to explain
these results

Open Repository and Bibliography - Luxembourg

 Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7   199 Fostering process skills with the educational technology software MathemaTIC in elementary schools Ben Haas1, Yves Kreis² and Zsolt Lavicza1 1Johannes Kepler University, Linz School of Education, Austria, ben.haas@outlook.com, ²University of Luxemburg, Luxemburg This study reports the use of automated tutoring and scaffolding implemented in the module “arithmetic word problem” in the educational technology software MathemaTIC in grade 3 (age 8 to 10). We examined 246 students with access to MathemaTIC and receiving tutoring and scaffolding through a one-to-one learning setting with this technology. The control group (n=226) had access to the same learning tasks and worked with paper-and-pencil without MathemaTIC but with their teachers. Results showed that the experimental group finished with higher outcome scores than the control group. This paper will outline the study and attempts to explain these results. Keywords: educational technology, process skills, elementary school, mathematics, problem-solving INTRODUCTION Teaching arithmetic word problems in grade 3 (age 8 to 10) is reported by the teachers in Luxemburg, as one of the most challenging topics in mathematics. Teachers in our study suggested that students struggle in class to solve arithmetic word problems, due to a lack of comprehension in reading, understanding of the wording and identifying the arithmetic operations to execute. In these tasks, students required process skills such as arguing, communicating, representing, and problem-solving (Selter & Zannetin, 2018). Moreover, based on the result of the national school monitoring EpStan in mathematics and language, students with low reading skills are also those who are low performing in mathematics (Sonnleitner et al., 2018). Similar to the findings of LeBlanc and Weber-Russel (1996), there is a connection between well-developed skills in reading and understanding of the mathematics course language and mastering process skills. Therefore, many students need continuous assistants from a teacher while learning to solve arithmetic word problems. In class, however, the group of students is heterogeneous, and a close follow-up is challenging to realize. In 2016, the Ministry of Education in Luxemburg developed, jointly with the Canadian company Vretta, an educational technology software for mathematics learning in elementary schools called MathemaTIC. A multidisciplinary team created a module inside MathemaTIC with an automated tutoring system to foster process skills in arithmetic word problems in grade 3 in order to create new learning possibilities and to address the low performances. The instructional design of the module aimed for one-to-one learning in class and at home for students without additional guidance from teachers or parents. We carried out a quantitative study to obtain findings on the use of  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7  200 this arithmetic word problem module by addressing the following two research questions: RQ1: Are students who learn process skills in arithmetic word problems with MathemaTIC likely to improve at the same degree compared to a traditional paper-and-pencil setting with the guidance of the teachers? RQ2: What are the limitations and opportunities of a one-to-one setting with MathemaTIC? Hence, we will present the design of the automated tutoring system and some results of the quantitative study in which we examined 246 students with access to MathemaTIC in a one-to-one learning setting without their teachers compared to a control group (n=226), that had access to the same learning items using paper-and-pencil, but worked with their teachers. THEORETICAL FRAMEWORK  Figure 1: Main view of the module “arithmetic word problem” The structure of the automated tutoring in the arithmetic word problem module was based on the Competence-Learning-Intervention-Assessment model by De Corte et al. (2004) and the Four-Component Instructional Design (4C-ID) model by van Merriënboer& Kester (2005) for learning complex skills. Both models suggested that students should learn through guided learning tasks and then apply the skills in tasks that are gaining in complexity and lowering in guidance. Furthermore, students should develop a cognitive structure applicable for these complex skills with meta-tools (Trouche, 2004) in the new tasks. Hence, in the module arithmetic word problems (compare figure 1), students started working on guided learning tasks (blue), followed  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7   201 by semi-guided tasks (red) and finally complex tasks without guidance (black). The scaffolding system, based on the multimedia learning theory (Mayer, 2005), consisted in listening to the wordings, interacting with the images and arithmetic operations, and self-regulating their solving process through (non-adaptive) guidance from a fictitious character (one for each of the four solving steps). The tasks were autocorrected, and direct feedback was given to the student. The different arithmetic tasks were addition and subtraction word problems as  recommended in the curriculum for grade 3 (MENFP, 2011), based on the criteria for constructing and solving arithmetic word problems (Franke & Ruwisch, 2010) and structured through the semantic classification of Vergnaud(1982) in transformative, compositional and comparative problems. The tasks were related to situations and places from the students’ living environment: “Luc does a bike tour from Luxembourg to Echternach with his 3 friends. The odometer on his bike is at 125 km at the start of his trip. The tour is 42 km long. What will the odometer show when they get to Echternach?”.  Figure 2: Guided use of a meta-tool: highlighting information and creating a scheme The first part of the module was dedicated to learning process skills (first 14 “blue” tasks after the key in figure 1). Students practised different process skills separately in guided learning tasks. These tasks then lead to discovery and manipulation of meta-tools supporting the different process skills (i.e. identifying relevant information in the wording with a highlighter tool and creating a resolution scheme, compare figure 2) to make it easier for students to solve the problem.  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7  202  Figure 3: “Black” item without guidance The second part focused on using the learned process skills in the identified arithmetic word problems (15 tasks organised as 3 ovals on the top of figure 1). Thus, students solved the different typologies of arithmetic word problems (combination, transformation and comparison) with the help of the learned meta-tools (compare menu bar on the left of figure 3). Each typology was presented in a set of three levels from guided tasks (blue), semi-guided tasks (red) to complex tasks (black). In the guided tasks (blue) in each typology, students needed to follow four steps solving procedure using the learned meta-tools: they analysed the wording, modulated the content into a resolution plan, executed the arithmetic operations and verified their results. In the semi-guided tasks (red), students were asked if they wanted to use the meta-tools, but could choose not to utilise them. In the complex tasks (black), they solved tasks with multiple arithmetic operations and no scaffolding was offered. They could use the meta-tools, but without additional guidance. METHODOLOGY In this section, we describe the quantitative pre-/post-test approach we utilised to measure if students in grade 3 (age 8 to 10) who learn arithmetic word problems with MathemaTIC in a one-to-one setting are likely to improve at the same degree compared to a traditional paper-and-pencil setting with teachers. The experimental group worked with the arithmetic word problem module in MathemaTIC in a one-to-one setting, without a specific teacher guidance. Their teachers did only ensure access to  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7   203 MathemaTIC and helped with technical issues. The control group did the same tasks using paper-and-pencil, however with the guidance and assistance of their teachers. Both groups worked for 20 hours (2 hours per week over a period of 10 weeks) on process skills in arithmetic word problems. During the study, we observed three different moments in the learning behaviour of the students within the experimental and control groups. Besides, we interviewed their teachers on their perception of the students’ learning with or without MathemaTIC based on the research questions RQ1 and RQ2. Participants of this quantitative study were 48 randomly selected classes with 667 students in grade 3 (age 8 to 10) in elementary schools in Luxemburg. We used the variables age, gender, and performance in mathematics of EpStan to identify matched pairs and assigned classes to experimental and control groups. We allocated 278 students to the control group, working using paper-and-pencil, and 389 students to the experimental group, working with the arithmetic word problems module in MathemaTIC. At the end of the study, 34 classes with 472 (8 with a missing post-test) students remained: 246 (2) students in the experimental group and 226 (6) students in the control group. The dropout was due to local technical errors (low WiFi signal or non-working hardware) while using MathemaTIC or simply because of a missing post-test for the whole class. Students in both groups performed an identical pre-test and post-test with 15 items based on the different typologies of arithmetic word problems (combination, transformation and comparison) with one or two operations and one item with a combinatorial problem. This combinatorial item allowed us to observe if students would transfer their learned skills into another typology of problem. Both tests have been created based on the experiences from author groups from the national school monitoring and based on the skills from the curriculum. RESULTS AND DISCUSSION Results from the experiment suggest that the use of the module “arithmetic word problem” in MathemaTIC is a promising approach to foster process skills in mathematics in a one-to-one setting. The statistical analysis below was carried out using the software R (R Core Team, 2020). Cronbach’s alpha (Revelle, 2019) indicates a good reliability for the pre-test (α=0.774) and the post-test (α=0.787). The detailed analysis shows that dropping one of the 16 items will only slightly increase the reliability for question 1 of the pre-test (α=0.777) and that there are no reverse-scored items. Thus, from this point of view, all items are to be kept in the tests. However, several questions in the pre-test (Q1: 0.16, Q3: 0.27, Q11: 0.24) as well as in the post-test (Q1: 0.26, Q3: 0.26) have an item-rest correlation below 0.3. Hence, they do not correlate very well with the scale overall. The success rates are 81% vs. 76% for question 1 (low difficulty level), 18% vs. 38% for question 3 (high difficulty level) and 3% vs. 9% for question 11 (very high difficulty level). Although extremely easy or difficult items only poorly allow to discriminate, they were  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7  204 needed to sample content and objectives adequately. Thus, we kept all items for further analysis.  Figure 4: Increase in score over time for the group We used lme4(Bates et al., 2015) to perform a linear mixed-effect analysis of the test result score predicted by the fixed effects time (pre-/post-test), control/experimental group and their interaction as well as the random effect student. Visual inspection of residuals plots revealed minor deviations from homoscedasticity and normality, which we accounted for by using bootstrapped confidence intervals. The main effect time has an estimate of 1.14 points (95% CI [0.56, 1.72]) for the control group. Thus, post-test scores of students from the control group were significantly higher than those in the pre-test. The main effect group has an estimate of -0.29 points (95% CI [-0.89, 0.28]) in the pre-test. On the one hand, both groups were comparable at the beginning of the study, because the confidence interval contains 0, and on the other hand, the experimental group probably had, in the pre-test, slightly lower test results than in the control group. Finally, the interaction effect time x group had an estimate of 0.90 points (95% CI [0.15, 1.68]). This effect underscores the fact that the performance gains of the experimental group, working with the educational software MathemaTIC, were significantly larger than those in the control group, resulting in somewhat better post-test performance although starting with a somewhat lower pre-test performance (compare figure 4). During the classroom observations and the interviews, we were able to collect data in both groups on the motivation, participation and transfer of skills. Thus, in the experimental group, teachers reported that students' motivations to solve and discuss  Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7   205 arithmetic word problems were higher than during the regular course (without MathemaTIC) and that they voluntarily exchanged on the tasks after the resolution. Teachers attributed the increase of motivation to the gamification aspect of MathemaTIC as well as the guidance and direct feedback given by the educational technology. According to teachers’ reports, some students suggested in other teaching hours (without MathemaTIC) to use the learned process skills to solve mathematical tasks (i.e.: Calculating the area of the classroom floor). In the control group, teachers stated that there was no change in motivation and some students had significant difficulties (i.e. understanding wording or findings of the arithmetic operation) to solve all the given tasks on paper.  CONCLUSION AND OUTLOOK Our findings highlighted that students in the experimental group improved their performances in arithmetic word problems significantly using the educational technology software MathemaTIC in one-to-one setting. Students learned meta-tools on process skills and successfully solved addition and subtraction word problems in all topologies without the direct guidance of a teacher or a parent. Teachers reported a high acceptance in class and an overall increase in motivation and participation of the students in mathematics courses. Thus, the module on arithmetic word problems in the educational technology software MathemaTIC is a viable alternative to the traditional paper-and-pencil course. Over time it could be a valuable asset to support students individually or in groups or even the entire class with MathemaTIC within traditional courses. We will perform further investigation on the fixed effects of gender, age, nationality, spoken language (L1) and the random effect school. Additionally, we will investigate all process skills in detail by performing a qualitative comparison of the pre-test and the post-test in our future analyses. Hopefully, we can further narrow the origin of the observed significant performance gains of the experimental group and we will report these analyses in future publications. ACKNOWLEDGEMENTS The Ministry of Education (MENJE) and the company Vretta are thanked for the collaboration and initiatives on the educational technology software MathemaTIC. The University of Luxemburg (LUCET) is thanked for the support in this research. REFERENCES Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1–48.  De Corte, E., Verschaffel, L., &Masui, C. (2004). The CLIA-model: A framework for designing powerful learning environments for thinking and problem solving. European Journal of Psychology of Education, 19(4), 365–384.   Proceedings of the 10th ERME Topic Conference MEDA 2020 - ISBN 978-3-9504630-5-7  206 Franke, M., &Ruwisch, S. (2010). Didaktik des Sachrechnens in der Grundschule (F. Padberg, Ed.; 2nd ed.). Springer Spektrum.  LeBlanc, M. D., & Weber-Russell, S. (1996). Text integration and mathematical connections: A computer model of arithmetic word problem solving. Cognitive Science, 20(3), 357–407.  Mayer, R. (2005). The Cambridge Handbook of Multimedia Learning (R. Mayer, Ed.). Cambridge UniversityPress. MENFP. (2011). Plan d’études: École fondamentale. Ministère de l’Éducation nationale et de la Formation professionnelle. http://www.men.public.lu/catalogue-publications/themes-transversaux/cen/cens/plan-etudes/fr.pdf R Core Team. (2020). R: A Language and Environment for Statistical Computing (4.0.1) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/ Revelle, W. (2019). psych: Procedures for Psychological, Psychometric, and Personality Research (1.9.12) [Computer software]. Northwestern University. https://CRAN.R-project.org/package=psych Selter, C., &Zannetin, E. (2018). Mathematik unterrichten in der Grundschule: Inhalte—Leitideen—Beispiele. Friedrich Verlag GmbH. Sonnleitner, P., Krämer, C., Gamo, S., Reichert, M., Muller, C., Keller, U., &Ugen, S. (2018). Schülerkompetenzen im Längsschnitt—Die Entwicklung von Deutsch-Leseverstehen und Mathematik in Luxemburg zwischen der 3. Und 9. Klasse. Luxembourg Centre for EducationalTesting, UniversitätLuxemburg; Service de Coordination de la Recherche et de l’Innovation pédagogiques et technologiques (SCRIPT). Trouche, L. (2004). Environnements Informatisés et Mathématiques: Quels usages pour quels apprentissages? Educational Studies in Mathematics, 55(1), 181–197.  van Merriënboer, J. J. G., & Kester, L. (2005). The Four-Component Instructional Design Model: Multimedia Principles in Environments for Complex Learning. In R. Mayer (Ed.), The Cambridge Handbook of Multimedia Learning (pp. 71–94). Cambridge University Press; Cambridge Core.  Vergnaud, G. (1982). A classification of cognitive tasks and operations of thought involved in addition and subtraction problems. Addition and Subtraction : A Cognitive Perspective, 39–59.   

Fostering process skills with the educational technology software MathemaTIC in elementary schools

https://orbilu.uni.lu/bitstream/10993/44538/1/Haas%20et%20al.%20-%202020%20-%20Fostering%20process%20skills%20with%20the%20educational%20tech.pdf

Fostering process skills with the educational technology software MathemaTIC in elementary schools

Abstract

Similar works

Full text

Available Versions

Open Repository and Bibliography - Luxembourg