Evaluating emotion visualizations using AffectVis, an affect-aware dashboard for students

Purpose: The purpose of this paper is to evaluate four visualizations that represent affective states of students. Design/methodology/approach: An empirical-experimental study approach was used to assess the usability of affective state visualizations in a learning context. The first study was conducted with students who had knowledge of visualization techniques (n=10). The insights from this pilot study were used to improve the interpretability and ease of use of the visualizations. The second study was conducted with the improved visualizations with students who had no or limited knowledge of visualization techniques (n=105). Findings: The results indicate that usability, measured by perceived usefulness and insight, is overall acceptable. However, the findings also suggest that interpretability of some visualizations, in terms of the capability to support emotional awareness, still needs to be improved. The level of students" awareness of their emotions during learning activities based on the visualization interpretation varied depending on previous knowledge of information visualization techniques. Awareness was found to be high for the most frequently experienced emotions and activities that were the most frustrating, but lower for more complex insights such as interpreting differences with peers. Furthermore, simpler visualizations resulted in better outcomes than more complex techniques. Originality/value: Detection of affective states of students and visualizations of these states in computer-based learning environments have been proposed to support student awareness and improve learning. However, the evaluation of visualizations of these affective states with students to support awareness in real life settings is an open issue.The work is partially supported by the eMadrid project (funded by the Regional Government of Madrid) under grant no S2013/ICE-2715, and the RESET project (Ministry of Economy and Competitiveness) under grant RESET TIN2014-53199-C3-1-R. The research is partially financed by the SURF Foundation of the Netherlands and the KU Leuven Research Council (Grant Agreement No C24/16/017, PDM16/044)

Why Feedback Literacy Matters for Learning Analytics

Learning analytics (LA) provides data-driven feedback that aims to improve learning and inform action. For learners, LA-based feedback may scaffold self-regulated learning skills, which are crucial to learning success. For teachers, LA-based feedback may help the evaluation of teaching effects and the need for interventions. However, the current development of LA has presented problems related to the cognitive, social-affective, and structural dimensions of feedback. In light of this, this position paper argues that attention needs to shift from the design of LA as a feedback product to one that facilitates a process in which both teachers and students play active roles in meaning-making. To this end, implications for feedback literacy in the context of LA are discussed.Comment: 8 pages. Accepted at the 2022 International Conference of the Learning Sciences (ICLS). https://2022.isls.org/proceedings

Evaluating emotion visualizations using AffectVis, an affect-aware dashboard for students

Purpose - The purpose of this paper is to evaluate four visualizations that represent affective states of students. Design/methodology/approach - An empirical-experimental study approach was used to assess the usability of affective state visualizations in a learning context. The first study was conducted with students who had knowledge of visualization techniques (n=10). The insights from this pilot study were used to improve the interpretability and ease of use of the visualizations. The second study was conducted with the improved visualizations with students who had no or limited knowledge of visualization techniques (n=105). Findings - The results indicate that usability, measured by perceived usefulness and insight, is overall acceptable. However, the findings also suggest that interpretability of some visualizations, in terms of the capability to support emotional awareness, still needs to be improved. The level of students’ awareness of their emotions during learning activities based on the visualization interpretation varied depending on previous knowledge of information visualization techniques. Awareness was found to be high for the most frequently experienced emotions and activities that were the most frustrating, but lower for more complex insights such as interpreting differences with peers. Furthermore, simpler visualizations resulted in better outcomes than more complex techniques. Originality/value - Detection of affective states of students and visualizations of these states in computer-based learning environments have been proposed to support student awareness and improve learning. However, the evaluation of visualizations of these affective states with students to support awareness in real life settings is an open issue

Focusing on the learner: charting a way forward for Australian education

This paper argues that Australia is unlikely to reach its goal of becoming a top 5 nation in education by 2025 without a major change in education policies.Executive summaryThe Australia in the Asian Century White Paper has set the laudable goal that: “By 2025, Australia will be ranked as a top five country in the world for the performance of our students in reading, science and mathematics literacy and for providing our children with a high-quality and high-equity education system”. This is a challenging goal and it demands that every child receives a first class education; however, this paper argues that the policies currently on the agenda will not deliver the standard of education required.The big picture for Australia’s education system is being held back by a confused and often incoherent debate. While discussion at the political level focuses on issues such as funding, public/private schooling, principal autonomy, performance pay, student and teacher tests and sector comparisons, policy makers risk oversimplifying teaching and missing the most important point.When it comes to achieving the Australia in the Asian Century White Paper’s goal, these issues are relatively irrelevant and lacking a strong evidence base on how they make an impact on student learning. Moreover, they imply a simplistic view of teaching as nothing more than information transmission and behaviour management, with an underlying message that for Australia’s education system to improve, teachers just need to work harder.This paper argues that teaching is far from simplistic but rather a complex, challenging, clinical practice profession that requires high calibre individuals. It outlines a way forward that has the potential to make a significant impact on the learning outcomes of all young Australians, focusing on the issues that matter: teachers and teaching

Learn Smarter, Not Harder – Exploring the Development of Learning Analytics Use Cases to Create Tailor-Made Online Learning Experiences

Our world is significantly shaped by digitalization, fostering new opportunities for technology-mediated learning. Therefore, massive amounts of knowledge become available online. However, concurrently these formats entail less interaction and guidance from lecturers. Thus, learners need to be supported by intelligent learning tools that provide suitable knowledge in a tailored way. In this context, the use of learning analytics in its multifaceted forms is essential. Existing literature shows a proliferation of learning analytics use cases without a systematic structure. Based on a structured literature review of 42 papers we organized existing literature contributions systematically and derived four use cases: learning dashboards, individualized content, tutoring systems, and adaptable learning process based on personality. Our use cases will serve as a basis for a targeted scientific discourse and are valuable orientation for the development of future learning analytics use cases to give rise to the new form of Learning Experience Platforms

Opening Up an Intelligent Tutoring System Development Environment for Extensible Student Modeling

ITS authoring tools make creating intelligent tutoring systems more cost effective, but few authoring tools make it easy to flexibly incorporate an open-ended range of student modeling methods and learning analytics tools. To support a cumulative science of student modeling and enhance the impact of real-world tutoring systems, it is critical to extend ITS authoring tools so they easily accommodate novel student modeling methods. We report on extensions to the CTAT/Tutorshop architecture to support a plug-in approach to extensible student modeling, which gives an author full control over the content of the student model. The extensions enhance the range of adaptive tutoring behaviors that can be authored and support building external, student- or teacher-facing real-time analytics tools. The contributions of this work are: (1) an open architecture to support the plugging in, sharing, re-mixing, and use of advanced student modeling techniques, ITSs, and dashboards; and (2) case studies illustrating diverse ways authors have used the architecture

Mobile-assisted language learning through learning analytics for self-regulated learning (MALLAS): A conceptual framework

Many adult second and foreign language learners have insufficient opportunities to engage in language learning. However, their successful acquisition of a target language is critical for various reasons, including their fast integration in a host country and their smooth adaptation to new work or educational settings. This suggests that they need additional support to succeed in their second language acquisition. We argue that such support would benefit from recent advances in the fields of mobile-assisted language learning, self-regulated language learning, and learning analytics. In particular, this paper offers a conceptual framework, mobile-assisted language learning through learning analytics for self-regulated learning (MALLAS), to help learning designers support second language learners through the use of learning analytics to enable self-regulated learning. Although the MALLAS framework is presented here as an analytical tool that can be used to operationalise the support of mobile-assisted language learning in a specific exemplary learning context, it would be of interest to researchers who wish to better understand and support self-regulated language learning in mobile contexts

Embracing imperfection in learning analytics

© 2018 Copyright held by the owner/author(s). Learning Analytics (LA) sits at the confluence of many contributing disciplines, which brings the risk of hidden assumptions inherited from those fields. Here, we consider a hidden assumption derived from computer science, namely, that improving computational accuracy in classification is always a worthy goal. We demonstrate that this assumption is unlikely to hold in some important educational contexts, and argue that embracing computational “imperfection” can improve outcomes for those scenarios. Specifically, we show that learner-facing approaches aimed at “learning how to learn” require more holistic validation strategies. We consider what information must be provided in order to reasonably evaluate algorithmic tools in LA, to facilitate transparency and realistic performance comparisons

License to evaluate: Preparing learning analytics dashboards for educational practice

Learning analytics can bridge the gap between learning sciences and data analytics, leveraging the expertise of both fields in exploring the vast amount of data generated in online learning environments. A typical learning analytics intervention is the learning dashboard, a visualisation tool built with the purpose of empowering teachers and learners to make informed decisions about the learning process. Related work has investigated learning dashboards, yet none have explored the theoretical foundation that should inform the design and evaluation of such interventions. In this systematic literature review, we analyse the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. Our analysis revealed that very few dashboard evaluations take into account the educational concepts that were used as a theoretical foundation for their design. Furthermore, we report findings suggesting that comparison with peers, a common reference frame for contextualising information on learning analytics dashboards, was not perceived positively by all learners. We summarise the insights gathered through our literature review in a set of recommendations for the design and evaluation of learning analytics dashboards for learners