Attributes of teaching and learning contexts provide rich information about how students participate in learning activities. By tracking and analyzing snapshots of these attributes captured continuously throughout the duration of the learning activities, teachers can identify individual students who need special attention and apply different pedagogical actions to them. This paper describes the results of the work-in-progress study in exploring characteristics of fine-grained behaviors of learning mathematics in tablet-based e-learning activities. An experimental platform called SkyApp is built. Through SkyApp, teachers can create e-learning activities and track learning records of students after the delivery of the activities. SkyApp supports capturing, storing and analyzing of fine-grained behaviors of students. Pilot tests have been done in two primary schools for eight months. The review of the tests demonstrates the potential in performing learning analytics. By applying clustering algorithms on multiple learning metrics of marks, time and number of attempts for students in solving mathematics questions, classification of students by learning characteristics of performance and engagement can be formulated. [For the complete proceedings, see ED579335.]published_or_final_versio

Chan, TY

Chu, SKW

Hui, CK

Kuo, YU

Ng, YL

Shum, KH

Yeung, CY

HKU Scholars Hub

Title Exploring Characteristics of Fine-Grained Behaviors of LearningMathematics in Tablet-Based E-Learning ActivitiesAuthor(s) Yeung, CY; Shum, KH; Hui, CK; Chu, SKW; Chan, TY; Kuo, YU;Ng, YLCitationInternational Association for Development of the InformationSociety (IADIS) International Conference on E-Learning, Lisbon,Portugal, 20-22 July 2017. Proceedings of the InternationalConference E-Learning 2017, p. 149-153Issued Date 2017URL http://hdl.handle.net/10722/245459Rights This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.EXPLORING CHARACTERISTICS OF FINE-GRAINED BEHAVIORS OF LEARNING MATHEMATICS IN  TABLET-BASED E-LEARNING ACTIVITIES Cheuk Yu Yeung1, Kam Hong Shum2, Lucas Chi Kwong Hui1, Samuel Kai Wah Chu2,  Tsing Yun Chan1, Yung Nin Kuo1 and Yee Ling Ng1 1Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong 2Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong ABSTRACT Attributes of teaching and learning contexts provide rich information about how students participate in learning activities. By tracking and analyzing snapshots of these attributes captured continuously throughout the duration of the learning activities, teachers can identify individual students who need special attention and apply different pedagogical actions to them. This paper describes the results of the work-in-progress study in exploring characteristics of fine-grained behaviors of learning mathematics in tablet-based e-learning activities. An experimental platform called SkyApp is built. Through SkyApp, teachers can create e-learning activities and track learning records of students after the delivery of the activities. SkyApp supports capturing, storing and analyzing of fine-grained behaviors of students. Pilot tests have been done in two primary schools for eight months. The review of the tests demonstrates the potential in performing learning analytics. By applying clustering algorithms on multiple learning metrics of marks, time and number of attempts for students in solving mathematics questions, classification of students by learning characteristics of performance and engagement can be formulated. KEYWORDS Fine-grained Behaviors, Tablet-based E-learning Activities, Learning Analytic, Learning Metrics, Learning Traits 1. INTRODUCTION The infrastructure of e-learning in recent years allows teachers, school administrators, students and parents to monitor learning records that are generated during the delivery of learning activities, through which the insights of learners’ behaviors can be obtained by learning analytics (Becker, 2013). Learning analytics (Siemens, 2010) is defined as the use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning. Based on the results of learning analytic, teachers can identify students with learning difficulties so that strategic interventions can be made (Dawson, 2010). Researchers explore different technology and data sources to discover digital footprints of learners, such as sensors for capturing and analyzing eye-tracking, heartbeat, skin conductivity, gesture, voice and etc. Blikstein (2013) called this approach as multimodal learning analytics. Through automated, fine-grained and high-frequency data collection, this approach allows measurement of affective factors which include emotions of the learners. Rienties and Rivers (2014) review more than 100 recent studies of learning analytics in which the role of emotions is considered as one of the key drivers of  e-learning. They identify approximately 100 different emotions that may have different levels of impact on learners’ attitude and cognition. In addition to learning emotion, researchers measure students’ behaviors of engagement (Richards, 2011) and motivation (Hershkovitz, & Nachmias, 2008) in e-learning activities.  RELATED WORK: As classified by Wise (2014), learning analytics can be applied by the stakeholders at macro and micro level. At macro level, learning analytics supports the decision making process of administrators that are related to the institutional level and beyond. Siemens and Long (2011) refer this type of learning analytics as Academic Analytics. Based on the records stored in learning management systems (LMS) and virtual learning environment (VLE), visual graphical tools are built to show activity information of students, which include frequency of using resources, time spent per student in each resource, etc. Notable International Conference e-Learning 2017149examples of learning analytics tools that explore the records in LMS such as Matep (Zorrilla & Álvarez, 2008). These platforms help monitoring learning records that are generated by the LMS that is adopted by the school for delivering learning activities. More recently, tools of learning analytics have been developed to perform analysis of log data in VLE (Agudo et al., 2013) and Khan Academy platform (Ruipérez et al., 2014). These analytic tools are often used to determine summative learning data and identify “at risk” students, but they are pedagogy-neutral. As for supporting teaching and learning at the micro level, tools require addressing the challenges of capturing, analyzing and displaying learning data for improving pedagogical practices. For example, LOCO-Analyst (Jovanović et al., 2008) obtains learning data from the online learning environment and presents the feedback data to teachers for evaluation.  However, it is a heavy burden for the teachers of primary and secondary schools if they need to carry out the works of design, development, and delivery of e-learning activities by themselves. To tackle this challenge, teachers need a tool to capture, analyze and present the learning data. There are currently very few tablet-based mobile apps that keep track of on the fine-grained data about students’ inputs during e-learning activities. This paper introduces a new app, called SkyApp, as a platform to explore the possibility of using learning analytic to enhance the outcome of teaching and learning. Some parts of our work have been presented in CITERS 2016 (Shum et al., 2016), and a detail technical report is posted on our university’s website (Hui et al., 2015). 2. METRICS OF LEARNING BEHAVIORS SkyApp aims at helping mathematics teachers to deliver tablet-based e-learning activities. It allows teachers to import teaching and assessment contents in developing e-learning activity simply by file upload or  photo-taking based on their existing teaching materials. The mathematics teachers of the participating schools create learning activities such as worksheets, quizzes, competitions, and homework for primary 4 to 5 (Ages 9-10) classes with the help of SkyApp. The objectives of these e-learning activities align with the curricular goals defined in the schools’ teaching plan and annual subject plan. By measuring students’ input on tablet computers, it is possible to examine fine-grained students’ behaviors in detail. SkyApp offers facilities for students to provide answers or responses to the questions or assessment by typing, handwriting and adding emojis. SkyApp captures inputs such as time spent on each part of the e-learning activity and the details of the handwriting which are essential in supporting data analysis.    Figure 1. Left - Entering Answer Box, Adding Emoji and Handwriting Inputs, Right - Worksheet 31 The facility of Answer Boxes (Figure 1 –Left) helps to capture the inputs of learners. SkyApp captures the answers entered by students and performs automatic matching with the modeled answers. In addition, students can select and add emojis to the SkyApp, to express common emotions in participating the  e-learning activity (Figure 1 - Left). In the form of worksheets presented in SkyApp, students can finish the questions under a specific topic, such as arithmetic operations of fraction, in each e-learning activity. Through the event logs of SkyApp, the inputs of users can be captured. There are two types of event logs, ISBN: 978-989-8533-63-0 © 2017150namely independent and dependent events. An independent event occurs only at one specific instance in time, for example inserting an emoji by a student is recorded as an independent event. A dependent event whereas records information at multiple instances of time, starting by a start of event action and following a series of in-process actions until ending by the end of event action. For example, during inputting answers in the Answer Boxes of the questions Q1 and Q2 in a worksheet, inputs are recorded as dependent events. Idle Times and Answer Times of a user can be measured in answering the first two questions of the worksheet (Figure 2).    Figure 2. Example of Idle Times and Answer Times The learning metrics of Time, Attempts and Marks are used to perform analysis of learning data. Based on event logs, the measuring metric of Time is formulated as the sum of Idle Times and Answer Times of all the questions in a worksheet. By taking the maximum number of attempts in answering the same question in a worksheet, the measuring metric of Attempts is formulated. For the measuring metric of Marks, it is determined by the sum of marks assigned in each correctly answered question. 3. DISCOVERING LEARNING BEHAVIORS BY CLUSTERING With the infrastructure of mobile learning in place, which includes WiFi network, tablet computers and educational mobile apps, primary and secondary schools can now adopt new models of e-learning that unleash the potential of applying new technology in education. Pilot tests are conducted with the participation of the mathematics teachers in the classes of two local primary schools. The tests were conducted mainly by a total of 64 students. All students are from one class of primary 4 and another class of primary 5 separately in the two primary schools. The test lasted 8 months in the academic year 2015-2016. The e-learning activities essentially include mathematics questions in the form of worksheets that are delivered as class exercises. To conduct the experiments, tablets of all the students and the teachers in the class are running SkyApp which are connected to the WiFi available in their schools. SkyApp is written in Objective-C that runs in iOS tablets. The data captured by SkyApp is structured in JSON format before sending to the back-end system for learning analytics. To perform analysis of the records captured, an analytical tool is built in Python to extract raw data from the database and convert them into CSV files. Data analytics can then be performed by calling statistical libraries through R programming.  Analysis reports and classification results are then presented graphically in the dashboard by HTML. Clustering algorithms (Rokach, 2009) are used to find the data points represented in specific learning metrics that group together into a set of clusters. Patterns of learning characteristics can be identified by applying clustering algorithms on one or multiple learning metrics. The clustering method is called Complete-linkage clustering, which is one of the agglomerative hierarchical clustering. Literally, agglomerative method means that each object starts out as its own cluster, and close individuals or clusters are gradually merged until all individuals are grouped into a single cluster or any certain termination conditions are satisfied. Four criteria are used for identifying the best number of clusters, namely Silhouette, KL, CH and C-Index. A scientific way to study the relationship of clusters is to calculate the Euclidean distance between the center points of the clusters. However, the most straightforward way is to generate a graph that can visually observe the distance between clusters. A typical experiment of one of the participating schools is described in this paper to demonstrate the effects of exploring fine-grained learning behaviors by using SkyApp in mathematics classes. Figure 3 shows the results of the students of the school in using SkyApp to complete Worksheet 31 (Figure 1 - Right) which International Conference e-Learning 2017151consists of 8 questions of fraction multiplication. Clustering algorithms are performed based on the metrics of Time, Attempts and Marks. The results in Figure 3 are presented in four quartiles of ranges of Marks, which are the most common ranges used by teachers and schools to represent students’ performance. The clusters identified in Figure 3 can be considered as meaningful patterns of learning characteristics of engagement and performance. As shown in Figure 3, Cluster D splits into two groups, one includes students 478, 484, & 503 and the other includes Students 497 & 499. The students in this Cluster scored relatively low marks, tried relatively fewer attempts and took relatively short time in answering the questions. These students show possible signs of relatively low learning engagement and performance. Students of Cluster A and Cluster E show efforts in making extra attempts and spending more time in answering the questions, whereas the students of Cluster B and Cluster C show relative strength in learning performance among other students. To further distinguish the difference between Clusters B and C, different ranges of Marks can be applied to the presentation of Figure 3. The benefits of clustering of students’ learning characteristics to teachers are twofold: teachers can have an overview picture of the learning outcomes of the whole class in each e-learning activity, and teachers can have insights of the traits of learning of individual students.    Figure 3. Clustering by Attempts and Time in different ranges of Marks 4. EVALUATION Based on the results of student surveys on 64 students at the end of test period, around 80% of students want to continue to use SkyApp in Math classes and around 69% of them agree that SkyApp makes the learning in Math classes more interesting to them. 10 students are interviewed to collect detailed comments on the usability of SkyApp. Besides the suggestions of new functions to SkyApp and improvements required to make SkyApp more effective, students expressed keen interests in the interviews that teachers can use SkyApp to offer incentives to them by giving out cartoon icons upon completion of e-learning activities. This shows that student engagement can potentially be enhanced by elements of gamification. ISBN: 978-989-8533-63-0 © 20171525. SUMMARY AND FUTURE WORK An experimental platform called SkyApp is developed to support tablet-based e-learning activities in learning mathematics with the capabilities in supporting learning analytics based on the fine-grained learners’ behaviors due to students’ inputs and responses. Initial efforts have been made in classifying learning characteristics of performance and engagement based on the results of the pilot tests. The next critical step is to investigate whether learning analytics can be applied to classify other learning characteristics such as motivation and emotion. Rigorous evaluations by quantitative and qualitative research methods are required. This study also brings up some questions that deserve further investigation: 1) Real-time learning analytics: the results show that it is technically feasible to perform clustering algorithms on learning metrics in real-time. The most time demanding part of the analysis is the computation time of running clustering algorithm and generating graphs. The time required is 6 to 10 seconds. It is feasible for teachers to view the results of the learning analytics right after each e-learning activity so that the results can help the teacher to revise the pedagogical actions of subsequent activities. 2) Personalized supports to students: through  real-time analytics, teachers can revise their pedagogies and introduce specific design elements such as gamification elements to engage students, and provide scaffolding supports according to the pace and style of learning of individual students. We believe that with more participation in using SkyApp by teachers, more concrete personalized e-learning pedagogical strategies can be discovered. REFERENCES Agudo-Peregrina, Á.F., et al., 2014. Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in human behavior, 31, pp.542-550. Becker, B., 2013. Learning analytics: Insights into the natural learning behavior of our students. Behavioral & Social Sciences Librarian, 32(1), pp.63-67. Blikstein, P., 2013, April. Multimodal learning analytics. In Proceedings of the third international conference on learning analytics and knowledge (pp. 102-106). ACM. Dawson, S., 2010. ‘Seeing’the learning community: An exploration of the development of a resource for monitoring online student networking. British Journal of Educational Technology, 41(5), pp.736-752. Hershkovitz, A. and Nachmias, R., 2008, June. Developing a log-based motivation measuring tool. In Educational Data Mining 2008. Hui, C. K., et al., 2015. A Mobile App Platform for Discovering Learning Profiles and Analytics. Technical Report  TR-2015-08, Department of Computer Science, The University of Hong Kong Jovanovic, Jelena, et al. 2008. LOCO-Analyst: semantic web technologies in learning content usage analysis. International journal of continuing engineering education and life long learning, 18(1), pp.54-76. Siemens, G. and Long, P., 2011. Penetrating the fog: Analytics in learning and education. EDUCAUSE review, 46(5), p.30. Rienties, B. and Rivers, B.A., 2014. Measuring and understanding learner emotions: Evidence and prospects. Learning Analytics Review, 1, pp.1-28. Richards, G., 2011. Measuring engagement: Learning analytics in online learning. electronic Kazan, 2011. Rokach, L., 2009. A survey of clustering algorithms. In Data mining and knowledge discovery handbook (pp. 269-298). Springer US. Ruipérez-Valiente, José A., et al., 2015. ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform. Computers in Human Behavior, 47, pp.139-148. Shum, K. H., et al., 2016. SkyApp: a tablet-based e-learning design tool for mathematics teachers to cater for learning diversity. In CITE Research Symposium, CITERS 2016. Siemens, G., 2010. What are learning analytics? Elearnspace, August 25, 2010. Wise, A.F., 2014, March. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 203-211). ACM. Zorrilla, M.E. and Álvarez, E., 2008, July. MATEP: Monitoring and analysis tool for e-learning platforms. In Advanced Learning Technologies, 2008. ICALT'08. Eighth IEEE International Conference on (pp. 611-613). IEEE. International Conference e-Learning 2017153

Exploring Characteristics of Fine-Grained Behaviors of Learning Mathematics in Tablet-Based E-Learning Activities

http://hub.hku.hk/bitstream/10722/245459/1/content1.pdf

Exploring Characteristics of Fine-Grained Behaviors of Learning Mathematics in Tablet-Based E-Learning Activities

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub