Augustijn-Beckers, P.W.M.

Oliveira, I.

Verkroost, M.J.

English

Time is an underdeveloped aspect of textbook reading analytics. We used the Living Textbook, a concept-based digital textbook, to collect tracking data during an introductory MSc course. In this paper, we compare first-time concept visits versus subsequent concept visits, the time when these concept visits occur and the duration of the concept visits. With the collected information, we want to optimize the length and order of the learning pathways. Although many concept visits are generated close to the exam at the end of the course, these are often re-visits, indicating a review before the exam rather than procrastination

University of Twente Research Information

122648th Annual Conference, Enschede, the Netherlands SHORT PAPERSTEMPERAL TRENDS IN TEXTBOOK TRACKING DATA P.W.M. Augustijn-Beckers1 University of Twente Enschede, The Netherlands M.J. Verkroost University of Twente Enschede, The Netherlands I. Oliveira University of Twente Enschede, The Netherlands Conference Key Areas: 10. E-learning, open and online learning, blended learning, virtual reality and 12. Niche & novel engineering education topics Keywords: Learning analytics, Digital Textbook, Temporal analysis ABSTRACT Time is an underdeveloped aspect of textbook reading analytics. We used the Living Textbook, a concept-based digital textbook, to collect tracking data during an introductory MSc course. In this paper, we compare first-time concept visits versus subsequent concept visits, the time when these concept visits occur and the duration of the concept visits. With the collected information, we want to optimize the length and order of the learning pathways. Although many concept visits are generated close to the exam at the end of the course, these are often re-visits, indicating a review before the exam rather than procrastination.   1 INTRODUCTION Learning analytics are widely used in higher education to monitor and predict student success [3,4], and for learning design purposes [2]. This paper explores the use of textbook tracking data, collected during an introductory course on Geographic Information Science (GIS). The digital textbook, called the Living Textbook (LTB), is a combination of a concept map and a wiki with additional learning functionalities, including sequential learning. Learning pathways are chains of concepts addressing a single learning objective that lead students through the reading assignments. They are comparable with a chapter in a regular book.                                                        1 Corresponding Author  P.W.M. Augustijn-Beckers  p.w.m.augustijn@utwente.nl 122748th Annual Conference, Enschede, the Netherlands SHORT PAPERSThe learning behaviour of students is often non-linear and does not always follow the pre-defined learning pathways due to difference in learning experience and learning styles [7]. The same user can show different reading behaviour during different phases of the course. Ogata et al. [6] distinguish “preview” and “review” behaviour, indicating reading before or after the teaching moment. Junco and Clem [4] developed an engagement index to measure the interaction of students with textbooks, including the number of days and sessions the student was active. The temporal dimension of learning analytics has been underexposed, according to [5]. This study adds the perspective of time into the analysis of tracking data. The question addressed in this paper is: “Which temporal patterns can we find in students’ use of the Living Textbook, and how can we use this information to optimize the learning pathways?” 2 METHODOLOGY 2.1 First time versus revisits and their timing and duration The temporal aspect of tracking data subdivides into several essential elements: How many times the student visits the concept (1) when a student visits a concept (2), and how long the concept is visited (3). We distinguish two types of concept visits: first-time visits and consecutive visits. The assumption is that study behaviour differs between first-time and subsequent visits. The second element is the moment in time during the course when the subsequent visit takes place. A subsequent visit may be a review during exam preparation, especially when these revisits take place shortly before the exam. If a subsequent visit takes place close to the teaching moment and this teaching moment is at the beginning of the course, students may still be familiarizing themselves with the tool and effectively the learning during the first visit is small.   Figure 1 (left) first time visits (blue) and subsequent visits (orange) to prepare for the exam (middle) first time visits (familiarize with tool) and subsequent study visits (orange) (right) first time visits (blue) and restudy visits (orange)  The duration of the concept visit may also give an indication of the type of visit. When this concept visit is relatively short this browsing behaviour could be associated with learning how to operate the tool, or moving through a learning pathway to reach another concept for review. When the duration of the concept visit is long, this can be associated with actual study time. We analyse at three different levels of aggregation: the course, the learning pathway and the concept. For the course level, we analyse all concept visits made by students, for learning pathways 122848th Annual Conference, Enschede, the Netherlands SHORT PAPERSwe analyse visits made by students to concepts that are included in a learning pathway, and at the concept level, we analyse visits of students for a given concept. A combination of a first-time visit at the moment of teaching with a review before the exam leads to a bimodal distribution of the concept visits over time. We check for bimodality via the coefficient of bimodality and the skewness of the distribution [1]. The bimodality coefficient was calculated using the R package Modes. The bimodality coefficient has a range of zero to one where a value greater than 0.56 suggests bimodality.   2.2 Tracking data The 103 students from a 3-week face-to-face introductory course on GIS were tracked after they provided permission to do so. Before the course, all students followed an instruction session introducing them to the LTB. Besides the LTB, students had access to a pdf version of a traditional textbook with the same content. Before the start of the course, teachers defined fourteen learning pathways - each of these learning pathways covered reading materials that correspond to approximately one day of education. Length of the learning pathways varied between 5 – 14 concepts with a mean of 9.7 concepts per learning pathway.  3 RESULTS 3.1 Temporal trends on Course level Figure 2 shows the number of concept visits per day. The number of visits stabilises at around 200 concept visits a day, but after September 18th, we see a strong increase leading to 1600 concept visits on September 22nd, the day before the exam. Around 50% of the total number of concept visits is generated in the last four days of the course. Our students seem to show a large tendency towards “last-minute study behaviour”. However, from this graph, it is not possible to see if these concept visits were generated by revisiting students or by new students.  Figure 2 Number of students using the Living Textbook per day of the course  3.2 Temporal trends at learning pathway level When we analyse the concept visits per learning pathway instead of at course level, we see that student visits per learning pathway are bimodal showing a peak at the time of lecturing and a peak at the end of the course (preparation for the exam). The 05001000150020002-9-2019 7-9-2019 12-9-2019 17-9-2019 22-9-2019 27-9-2019Number of Concept hitsDays of the course 122948th Annual Conference, Enschede, the Netherlands SHORT PAPERSbimodality coefficient ranges from 0.668 - 0.896, indicating bimodality for all learning pathways. To analyse the difference between first-time and subsequent visits, we can look at the total number of visits. We seem to generate many subsequent visits (Figure 3 - blue) later in the course. This means that many students review the materials again in the days before the exam. When we compare a learning pathway scheduled early in the course (Figure 3 – path 8) with learning pathways scheduled later in the course (Figure 3 – path 138), we see a gradual shift in the distribution of new users in time. For a learning pathway scheduled later in the course (138), we see that the number of new users during the last days of the course is considerably larger. The time during the course when a learning pathway is scheduled seems to have a large impact on the timing of the concept visits.  Figure 3 First time visits versus re-visits per learning pathway  3.3 Time per concept We compare the median time spent per concept per learning pathway with the average time spent, for both new visitors and re-visitors (Figure 4). When we evaluate the median time spent for learning pathway 8 by first-time visitors, the results are very stable. Most time is spent on the first concept; the mean duration of a first-time concept visit of all other concepts is stable at about 1 minute (60 seconds). This pattern changes in subsequent visits. Concepts earlier in the learning pathway (to the left of the graph) get more attention compared to concepts later in the learning pathway (right of the graph). When we compare this to the average time spent, we see more variation between the duration of the concept visits. Concepts two and three seem to be most studied by first time users, and by subsequent users. 4 SUMMARY AND ACKNOWLEDGMENTS The total number of visits per day of the course is a very misleading number to base conclusions on. We see large peaks before the exam, but many of these concept visits were re-visits by students that earlier studied these concepts. The later a learning pathway is scheduled in the course, the more first-time concept visits are generated close to the exam. For early teaching, there is enough time to review the materials. Teachers should take this into account when designing their courses. The same applies to the order of concepts in the learning pathways. The earlier a concept is positioned in this learning pathway, the longer the time spent by students 123048th Annual Conference, Enschede, the Netherlands SHORT PAPERSin studying these topics, especially for subsequent visits. Further analysis can help to determine the preferred length of the learning pathways.     Figure 4 Example of the number of first time versus revisits – learning path with the number of visits (left) median time spent in seconds (middle) and the average time spent in seconds (right)  This research was made possible via a contribution from 4TU and by students that allowed us to track their navigation behaviour. REFERENCES [1] Ellison, A. (1987). Effect of Seed Dimorphism on the Density-Dependent Dynamics of Experimental Populations of Atriplex triangularis (Chenopodiaceae). American Journal of Botany, 74(8), 1280-1288.  [2] Foung, D., & Chen, J. (2019). A Learning Analytics Approach to the Evaluation of an Online Learning Package in a Hong Kong University. The Electronic Journal of E-Learning, 17(1), 11–24.  [3] Herodotou, C., Hlosta, M., Boroowa, A., Rienties, B., Zdrahal, Z., & Mangafa, C. (2019). Empowering online teachers through predictive learning analytics. 50(6), 3064-3079. doi:10.1111/bjet.12853 [4] Junco, R., & Clem, C. (2015). Predicting course outcomes with digital textbook usage data. Internet and Higher Education, 27, 54-63.  [5] Knight, S., Friend Wise, A., & Chen, B. (2017). Time for Change: Why Learning Analytics Needs Temporal Analysis. Journal of Learning Analytics, 4(3), 7–17. doi:https://doi.org/10.18608/jla.2017.43.2 [6] Ogata, H., Oi, M., Mohri, K., Okubo, F., Shimada, A., Yamada, M., . . . Hirokawa, S. (2017). Learning Analytics for E-Book-Based Educational Big Data in Higher Education. In H. Yasuura, C.-M. Kyung, Y. Liu, & Y.-L. Lin (Eds.), Smart Sensors at the IoT Frontier (pp. 327-350). Cham: Springer International Publishing. [7] Rootzén, H. (2015). Individualized learning through non-linear use of learning objects: With examples from math and stat. . Paper presented at the Proceedings of the European Conference on E-Learning, ECEL.  

Temperal trends in textbook tracking data

https://ris.utwente.nl/ws/files/256102564/Augustijn_Beckers2020temperal.pdf

Temperal trends in textbook tracking data

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information