Recently it has become increasingly difficult for teachers to get a grip on student’s learning experience, especially in the ear of distance-learning. This study analyzes teachers’ experiences in using an online monitoring tool of students’ metacognitive states. Through the open-ended questionnaire, we collected teachers’ reasoning to start using this new tool. We elaborated on the effect of this on teaching and learning and compared the official static course feedback with the dynamic provided by the tool. Our results show that teachers use this tool to understand student learning status more deeply and adjust the course according to the feedback. Moreover, teachers reported that they have been getting better quality summative course feedback after starting to use the tool.Non peer reviewe

Kivimäki, Ville Antero

Remes, Heikki

Romanoff, Jani

English

Helsingin yliopiston digitaalinen arkisto

https://helda.helsinki.fiBenefits of Online Course Diary Tool on Teaching and Learning ExperienceKivimäki, Ville Antero2021-06-09Kivimäki , V A , Romanoff , J & Remes , H 2021 , ' Benefits of Online Course Diary Tool onTeaching and Learning Experience ' , EUNIS 2021 A NEW ERA OF DIGITALTRANSFORMATION , Athens , Greece , 09/06/2021 - 11/06/2021 pp. 1-5 . <https://www.eunis.org/eunis2021/wp-content/uploads/sites/18/2021/05/EUNIS_2021_paper_30.pdf>http://hdl.handle.net/10138/355471acceptedVersionDownloaded from Helda, University of Helsinki institutional repository.This is an electronic reprint of the original article.This reprint may differ from the original in pagination and typographic detail.Please cite the original version.Benefits of Online Course Diary Tool on Teaching and Learning Experience Ville Kivimäki1*, Jani Romanoff2 and Heikki Remes2 1 University of Helsinki, Finland 2Aalto University, Finland Ville.Kivimaki@helsinki.fi, Jani.Romanoff@aalto.fi, Heikki.Remes@aalto.fi Abstract Recently it has become increasingly difficult for teachers to get a grip on student’s learning experience, especially in the ear of distance-learning. This study analyzes teachers’ experiences in using an online monitoring tool of students’ metacognitive states. Through the open-ended questionnaire, we collected teachers’ reasoning to start using this new tool. We elaborated on the effect of this on teaching and learning and compared the official static course feedback with the dynamic provided by the tool. Our results show that teachers use this tool to understand student learning status more deeply and adjust the course according to the feedback. Moreover, teachers reported that they have been getting better quality summative course feedback after starting to use the tool. 1 Introduction Borokhovski, Bernard, Tamim & Schmid (2016) found a low but significant positive effect in learning outcomes with learning technology use. The highest effect was found on feedback strategies. Furthermore, they conclude promoting technologies that support student’s cognitive processes. It is also clear that technology alone is not the solution. During the increasing digitalization of our remote education (including the effect of the covid-19 outbreak), we should have several actions of monitoring teaching and learning activities, as suggested by Alrefaie, Hassanien & Al-Hayani (2020). As indicated by Geraldine, Schalk, Rooney and Lang (2021), students prefer their data to first improve teaching quality and second, to improve course design. To tackle these challenges, we have developed the Course Diaries -online feedback tool; see Kivimäki et al. (2018). The concept was developed further from a theoretical perspective by Kivimäki  * https://orcid.org/0000-0003-4939-7400   et al. (2019), and effects on self-regulation and school engagement were carried out by Pesonen et al. (2020). Today the tool is available for a Moodle digital learning platform and Android and iOS mobile platforms. This two-interface system was selected based on the findings in our student usability research (Hahto, 2020). Previous research around the platform has focused on student perspectives and learning theories. In this paper, we focus on teachers’ views on a tool designed to support cognitive learning processes through metacognition. Our research questions are related to the 1) goals the teachers wanted to achieve by using the platform, 2) the changes teachers experienced while using the tool, and 3) the effect of the tool on the regular course feedback. 2 Methods Course Diaries is a learning analytics tool developed by Kivimäki et al. (2018) in a medium-size European university. The tool lists student’s courses and their intended learning outcomes as simplified short topics. On each topic, a student can assess his/her learning experience weekly through three fixed scale items and three open-response items: competence, difficulty, feeling, expectations, notes, reflection; see Figure 1. Students can choose between two user interfaces (Figure 2). Group level aggregated data is visualized in the Moodle application (Figure 1).   We created a full anonym online questionnaire with four open-ended questions for the teachers. Three of these are reported in this study: goals for the use of the CD tool, observed changes during the course, and effects on regular course feedback after the course. Our methodological approach is based on grounded theory (Glaser & Strauss, 1967). Based on this, we prepared the data for qualitative analysis by creating categories based on our observations from the data and coding the answers accordingly into these categories. Figure 1: Student’s mobile app UI (left) and Teacher’s report view in Moodle (right) Figure 2: General overview of dataflows for teacher and student UI 3 Results The current research on Course Diaries utilization covers around 40 courses by twelve different (N=12) teachers and around 2,000 students. The web questionnaire remained open for one week period. Seven teachers participated (n=7), and all participants answered all four questions of the questionnaire. Based on our findings, Course Diaries was mainly adopted as a part of instructional design to monitor students’ learning experience and help a teacher to better prepare for his/her teaching activities (Table 1). Also, two teachers reported on their aims to promote deeper learning.  Teachers observed four types of changes in their courses. Through deeper insights on what students think about course topics, teachers reported adapting course contents during the course. Two teachers reported increased thinking about their teaching, i.e., teacher’s metacognition. One reported noticing changes in students’ metacognition. In their answers related to the effects of regular course feedback, teachers reported having experienced mainly positive effects (n=4) or suggest the system be taken into wider use (n=2).  Table 1: Teacher’s goals for the tool use, observed changes and effect on regular course feedback Question type  Category Count Explanation Goals  Student monitoring 5 Teacher aims to monitor students in order to know how they are doing. Goals  Preparing for teaching  4 Teacher aims to use the data for preparing better in teaching activities. Goals  Promoting deeper learning 2 Teacher aims to promote deeper learning on his/her course. Changes  Adapted course content 4 Teacher reports changes on his/her course. Changes  Insights  3 Teacher reports getting better insights on course situation. Changes  Teacher’s metacognition 2 Teacher reports changes in his/her own thinking as a teacher. Changes  Student’s thinking 1 Teacher reports changes in student behavior. Feedback  Positive effects 4 Teacher reports getting better regular course feedback Feedback  Extremely positive  2 Teacher suggest the system to be taken into a wider use in the university. Feedback  No effects 1 Teacher reports no changes to regular feedback  4 Discussion and Conclusions Results show that teachers aim to use CD for student monitoring, being better prepared for instructional events, and some teachers want to promote deeper learning.  In observed changes, teachers report seeing results they were seeking. Besides, several teachers report thinking teaching on meta-level. Based on our results, there were no observed adverse effects on regular after the course feedback. Instead, teachers reported having positive effects on the regular course feedback, e.g., better overall course assessment due to teacher showing interest towards student feedback throughout the course. Some teachers extended their view to more extensive implementation of CD as a part of the university’s regular feedback system.  Prior research had focused on student perspective. The need for feedback activities during the course has been evident (Hahto, 2020). Mind mapping diary-solution brought too heavy workload on students (Pesonen et al., 2020). However, our student-friendly mobile application has been well adopted by students (Hahto, 2020). In this study, we have analysed teachers’ answers to an open-ended questionnaire. Their answers are of good quality, and they tell a coherent story that the CD solution is working very well.  As we continue to depend evermore on virtual platforms, we need to find effective methods to give teachers tools to get timely feedback on the course’s emotional climate and whether the students understand the taught topics. In a typical lecture situation, an experienced teacher would trust his/her gut feeling. Learning diary type of activity, such as Course Diaries, could be the modern solution here. References  Alrefaie, Z., Hassanien, M., & Al-Hayani, A. (2020). Monitoring Online Learning During COVID-19 Pandemic; Suggested Online Learning Portfolio (COVID-19 OLP). MedEdPublish, 9(1), 110, https://doi.org/10.15694/mep.2020.000110.1. Borokhovski, E. F., Bernard, R. M., Tamim, R. M., & Schmid, R. F. (2016). Technology integration in postsecondary education: A summary of findings from a set of related meta-analyses. Russian Psychological Journal, 13(4), 284-302. Geraldine, G., Schalk, A., Rooney, P., & Lang, C. (2021). A Stakeholder Informed Professional Development Framework to Support Engagement with Learning Analytics. In LAK21: 11th International Learning Analytics and Knowledge Conference (LAK21), April 12–16, Irvine, CA, USA. Conference presentation. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Strategies for qualitative research. Chicago, IL: Aldine. Kivimäki, V., Pesonen, J., Romanoff, J., Remes, H., & Kauppinen, T. (2018). Supporting understanding of students’ learning via visual self-assessment. EUNIS Annual Congress. Paris. Kivimäki, V., Pesonen, J., Romanoff, J., Remes, H., & Ihantola, P. (2019). Curricular Concept Maps as Structured Learning Diaries: Collecting data on self-regulated learning and conceptual thinking for learning analytics applications. Journal of Learning Analytics, 6(3), 106-121. https://doi.org/10.18608/jla.2019.63.13. Hahto, T. (2020). Student Experience of Mobile Dynamic Feedback System Application. Master's Thesis. Aalto University. Pesonen, J., Ketola, E. E., Kivimäki, V., & Ihantola, P. (2020). Does Using Structured Learning Diaries Affect Self-regulation or Study Engagement? An Experimental Study in Engineering Education. Conference proceedings - Frontiers in Education Conference (p. https://doi.org/10.1109/FIE44824.2020.9274163). Uppsala: IEEE. 5 Author biographies Ville Kivimäki is a Ph.D. student and working as a data science specialist at the University of Helsinki. He was a project manager in an online learning pilot that resulted into a learning analytics product Course Diaries at Aalto University. He graduated as a Master of Arts from Tampere University (2007). He has later deepened academic background through university pedagogical studies, programming and data science. He has worked in Tampere University, as Research Curator,  and Aalto University, as a teacher, Planning Officer, Team Leader and project manager in several administrative positions related to curriculum work, pedagogical development, student success and retention. LinkedIN: www.linkedin.com/in/ville-kivimaki Jani Romanoff is a professor of marine technology at Aalto University, School of Engineering. He received his D.Sc. degree from Helsinki University of Technology at 2007. He has a vast background in research of marine structures and computational design methods and received World Cultural Council Young Researcher Award (2014). He also has been active on developing international M.Sc. programmes in marine technology which are based on Problem-Based-Learning and career-oriented coaching of students towards life-long-learning. The programme is internationally recognised and Prof. Romanoff received Aalto School of Engineering Teaching Award from it on 2012. Prof. Romanoff has educated 8 Doctors of Science and 7 postdocs of which 1 holds professorship today. He has also instructed over 50 M.Sc. theses and 20 B.Sc. theses. This instruction is strongly connected to research in which he has published over 130 scientific journal and conference papers with his colleagues, mentees and students.  Heikki Remes is a professor in Marine Technology and a vice director of Master's Programme in Mechanical Engineering at Aalto University. Prof. Remes has 15 years teaching experience. He is educated six doctor of science and supervised about 40 Master’s and 20 Bachelor theses. Prof. Remes has participated in the development of two new Master's Programmes as well as Marine Technology curriculum. He is a member of Teaching Competence Assessment Group of Aalto University School of Engineering. He has received teaching award from the development of Shipbuilder’s Learning Portfolio. He has published over 100 scientific journal and conference papers. 

Benefits of Online Course Diary Tool on Teaching and Learning Experience

https://helda.helsinki.fi/bitstream/10138/355471/1/EUNIS_2021_paper_30.pdf

Benefits of Online Course Diary Tool on Teaching and Learning Experience

Abstract

Similar works

Full text

Available Versions

Helsingin yliopiston digitaalinen arkisto