Learning Analytics Dashboard for Teaching with Twitter

As social media takes root in our society, more University instructors are incorporating platforms like Twitter into their classroom. However, few of the current Learning Analytics (LA) systems process social media data for instructional interventions and evaluation. As a result, instructors who are using social media cannot easily assess their students’ learning progress or use the data to adjust their lessons in real time. We surveyed 54 university instructors to better understand how they use social media in the classroom; we then used these results to design and evaluate our own Twitter-centric LA dashboard. The overarching goals for this project were to 1) assist instructors in determining whether their particular use of Twitter met their teaching objectives, and 2) help system designers navigate the nuance of designing LA dashboards for social media platforms

Evaluating emotion visualizations using AffectVis, an affect-aware dashboard for students

Purpose: The purpose of this paper is to evaluate four visualizations that represent affective states of students. Design/methodology/approach: An empirical-experimental study approach was used to assess the usability of affective state visualizations in a learning context. The first study was conducted with students who had knowledge of visualization techniques (n=10). The insights from this pilot study were used to improve the interpretability and ease of use of the visualizations. The second study was conducted with the improved visualizations with students who had no or limited knowledge of visualization techniques (n=105). Findings: The results indicate that usability, measured by perceived usefulness and insight, is overall acceptable. However, the findings also suggest that interpretability of some visualizations, in terms of the capability to support emotional awareness, still needs to be improved. The level of students" awareness of their emotions during learning activities based on the visualization interpretation varied depending on previous knowledge of information visualization techniques. Awareness was found to be high for the most frequently experienced emotions and activities that were the most frustrating, but lower for more complex insights such as interpreting differences with peers. Furthermore, simpler visualizations resulted in better outcomes than more complex techniques. Originality/value: Detection of affective states of students and visualizations of these states in computer-based learning environments have been proposed to support student awareness and improve learning. However, the evaluation of visualizations of these affective states with students to support awareness in real life settings is an open issue.The work is partially supported by the eMadrid project (funded by the Regional Government of Madrid) under grant no S2013/ICE-2715, and the RESET project (Ministry of Economy and Competitiveness) under grant RESET TIN2014-53199-C3-1-R. The research is partially financed by the SURF Foundation of the Netherlands and the KU Leuven Research Council (Grant Agreement No C24/16/017, PDM16/044)

A Review of Tools and Techniques for Data-Enabled Formative Assessment

The purpose of this literature review is to understand the current state of research on tools that collect data for the purpose of formative assessment. We were interested in identifying the types of data collected by these tools, how these data were processed, and how the processed data were presented to the instructor or student for the purpose of formative assessment. We identified two categories of data: machine graded and activity stream data. The data were processed using three methods: unprocessed activity streams, descriptive data analysis, and data mining. Processed data were presented to students through reports and real-time feedback, and to instructors through reports and visual dashboards

The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms

Recently, the HCI community has seen increased interest in the design of teaching augmentation (TA): tools that extend and complement teachers' pedagogical abilities during ongoing classroom activities. Examples of TA systems are emerging across multiple disciplines, taking various forms: e.g., ambient displays, wearables, or learning analytics dashboards. However, these diverse examples have not been analyzed together to derive more fundamental insights into the design of teaching augmentation. Addressing this opportunity, we broadly synthesize existing cases to propose the TA framework. Our framework specifies a rich design space in five dimensions, to support the design and analysis of teaching augmentation. We contextualize the framework using existing designs cases, to surface underlying design trade-offs: for example, balancing actionability of presented information with teachers' needs for professional autonomy, or balancing unobtrusiveness with informativeness in the design of TA systems. Applying the TA framework, we identify opportunities for future research and design.Comment: to be published in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 17 pages, 10 figure

Design of interactive visualization of models and students data

This document reports the design of the interactive visualizations of open student models that will be performed in GRAPPLE. The visualizations will be based on data stored in the domain model and student model, and aim at supporting learners to be more engaged in the learning process, and instructors in assisting the learners

Evaluating emotion visualizations using AffectVis, an affect-aware dashboard for students

Purpose - The purpose of this paper is to evaluate four visualizations that represent affective states of students. Design/methodology/approach - An empirical-experimental study approach was used to assess the usability of affective state visualizations in a learning context. The first study was conducted with students who had knowledge of visualization techniques (n=10). The insights from this pilot study were used to improve the interpretability and ease of use of the visualizations. The second study was conducted with the improved visualizations with students who had no or limited knowledge of visualization techniques (n=105). Findings - The results indicate that usability, measured by perceived usefulness and insight, is overall acceptable. However, the findings also suggest that interpretability of some visualizations, in terms of the capability to support emotional awareness, still needs to be improved. The level of students’ awareness of their emotions during learning activities based on the visualization interpretation varied depending on previous knowledge of information visualization techniques. Awareness was found to be high for the most frequently experienced emotions and activities that were the most frustrating, but lower for more complex insights such as interpreting differences with peers. Furthermore, simpler visualizations resulted in better outcomes than more complex techniques. Originality/value - Detection of affective states of students and visualizations of these states in computer-based learning environments have been proposed to support student awareness and improve learning. However, the evaluation of visualizations of these affective states with students to support awareness in real life settings is an open issue

Modelo de diagnóstico de dificuldades de aprendizagem orientado a conceitos

The use of Virtual Learning Environments (VLE) in education has grown considerably in recent years, much due to the expansion of courses in distance mode. Such learning support spaces require you to think of new educational methods, particularly as regards the assessment of learning. Because of the large number of students in this type of education working in AVA, a large volume of data is generated. So to get yourself a good learning evaluation model, which offers the teacher possibilities for measuring student performance, it is necessary an analysis of such data. Moodle provides the teacher reports, charts and graphs that let you see the data for the actions of the students. Such actions represent from access to resources and materials, to participate in activities such as discussion forums and results of participation in questionnaires, for example. However, these Moodle native views do not take into account the real needs of teachers, especially in regard to effective monitoring of learning. Given the above, it is evident the need to have tools that help in this process. For this, there have to be a model permits, agile and flexible, storage and use of educational data of students to the application of techniques of Learning Analytics - measurement, collection, analysis and reporting of data on students and their contexts, for understanding and learning in order to optimize the environments in which they occur - with focus on diagnosis of learning disability situations in the context of the Distance Education. To evaluate this proposal, the ConcetpVis tool was implemented from the model proposed in this paper. Then, there was a case study in the Elementary Mathematics discipline of the Bachelor's Degree in Computer Education Unit Distance UFPB and finally presented a questionnaire to a group of teachers answered from his impressions.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPESO uso de Ambientes Virtuais de Aprendizagem (AVA) na educação tem crescido bastante nos últimos anos, muito em virtude da expansão dos cursos na modalidade à distância. Tais espaços de suporte a aprendizagem exigem que se pense em novos métodos educativos, sobretudo no que se refere à avaliação da aprendizagem. Em virtude da grande quantidade de alunos nessa modalidade de educação atuando no AVA, um grande volume de dados é gerado. Assim, para obter-se um bom modelo de avaliação de aprendizagem, que ofereça ao professor possibilidades de medir o desempenho dos alunos, faz-se necessário uma análise desses dados. O Moodle oferece ao professor relatórios, tabelas e gráficos que permitem visualizar os dados referentes às ações dos alunos. Tais ações representam desde o acesso a recursos e materiais didáticos, até a participação em atividades, como fóruns de discussão e resultados de participação em questionários, por exemplo. Contudo, essas visualizações nativas do Moodle não levam em consideração as reais necessidades dos docentes, sobretudo em relação a um acompanhamento efetivo da aprendizagem. Diante do exposto, fica evidente a necessidade de se ter ferramentas que auxiliem neste processo. Para isso, há de ter-se um Modelo que permita, de modo ágil e flexível, o armazenamento e a utilização dos dados educacionais dos alunos para a aplicação de técnicas de Learning Analytics – medição, coleta, análise e comunicação de dados sobre os alunos e seus contextos, para fins de compreensão e aprendizagem com fim de otimizar os ambientes em que ocorrem – com foco no diagnóstico de situações de dificuldade de aprendizagem no contexto da Educação a Distância. Para avaliar esta proposta, a ferramenta ConcetpVis foi implementada a partir do modelo proposto no presente trabalho. Em seguida, realizou-se um estudo de caso na disciplina Matemática Elementar do curso de Licenciatura em Computação da Unidade de Educação a Distância da UFPB e, por fim, apresentou-se um questionário para que um grupo de professores respondesse a partir de suas impressões

How did the e-learning session go? The student inspector

Good teachers know their students, and exploit this knowledge to adapt or optimise their instruction. Traditional teachers know their students because they interact with them face-to-face in classroom or one-to-one tutoring sessions. In these settings, they can build student models, i.e., by exploiting the multi-faceted nature of human-human communication. In distance-learning contexts, teacher and student have to cope with the lack of such direct interaction, and this must have detrimental effects for both teacher and student. In a past study we have analysed teacher requirements for tracking student actions in computer-mediated settings. Given the results of this study, we have devised and implemented a tool that allows teachers to keep track of their learners'interaction in e-learning systems. We present the tool's functionality and user interfaces, and an evaluation of its usability

L'utilisation du tableau de bord des activités d'apprentissage "Ma réussite" en formation en ligne à l'Université Laval pour soutenir l'autorégulation de l'apprentissage et la réussite

Malgré sa popularité grandissante, la formation en ligne en contexte universitaire doit composer avec des problématiques liées à la persévérance et à la réussite, puisque les taux d'abandon et d'échec seraient plus élevés dans ce type de formation qu'en présentiel sur le campus. Dans l'optique de soutenir la persévérance et la réussite des étudiants, l'Université Laval a développé un tableau de bord d'analyse des activités d'apprentissage qui vise à faire prendre conscience aux étudiants de leur participation dans leurs cours, et à les orienter vers des ressources d'aide. Cette recherche porte sur l'utilisation de ce tableau de bord (appelé Ma réussite) pour soutenir l'autorégulation de l'apprentissage et la réussite des étudiants de premier cycle en formation en ligne. Plus précisément, elle a pour objectif de documenter comment et par qui le tableau de bord est utilisé ainsi que les représentations des étudiants à son égard. Cette thèse vise également à vérifier si la théorie unifiée d'acceptation et d'utilisation de la technologie (UTAUT) peut expliquer l'utilisation du tableau de bord, et si son utilisation est liée à la réussite des étudiants. Pour ce faire, nous avons réalisé un sondage auprès de 309 étudiants de 15 cours en ligne différents dans quatre facultés. Nous avons également obtenu les traces numériques d'utilisation de Ma réussite par les étudiants durant la session d'automne 2019. Nos résultats indiquent que Ma réussite est peu utilisé, et que le nombre de visites est significativement plus élevé pour les étudiants qui suivent leur premier cours en ligne, pour ceux qui sont en contact avec l'outil pour la première fois ainsi que pour les primo-entrants. À l'aide du modèle d'autorégulation de Zimmerman (1998), nous avons exploré les perceptions des étudiants concernant l'utilité du tableau de bord pour soutenir l'autorégulation de l'apprentissage. Nos résultats indiquent que les étudiants sont d'accord que Ma réussite soutient l'autorégulation de l'apprentissage, notamment le monitorage et la régulation de l'effort. Les primo-entrants et les étudiants qui sont en contact avec l'outil pour la première fois perçoivent davantage le tableau de bord comme un soutien à l'autorégulation. Nous avons également documenté les perceptions des étudiants concernant les variables qui prédisent l'utilisation selon l'UTAUT, soit la performance attendue, l'effort attendu, l'influence sociale, les conditions facilitatrices et l'intention d'utilisation. Nos résultats indiquent que Ma réussite est utile, facile à utiliser et que les conditions facilitant son utilisation sont en place. Cependant, l'influence sociale pour l'utiliser est perçue comme étant faible. Les étudiants en contact avec l'outil pour la première fois et les primo-entrants perçoivent Ma réussite plus utile et ont davantage l'intention de l'utiliser à l'avenir que les autres étudiants. Ensuite, nous avons validé l'UTAUT, adaptée à l'étude, avec des analyses d'équations structurelles basées sur les moindres carrés partiels. Nos résultats ont permis de valider les hypothèses comme quoi la performance attendue, l'effort attendu, l'influence sociale et l'utilisation actuelle influencent l'intention d'utilisation. Cependant, nos résultats indiquent que les conditions facilitatrices ne sont pas un facteur déterminant de l'intention d'utilisation. Notre modèle de recherche permet d'expliquer 62 % de la variance de l'intention d'utilisation et 72 % de la variance de la performance attendue, avec comme seule variable prédictive le soutien à l'autorégulation. Finalement, nos résultats n'ont pas permis de confirmer un lien entre l'utilisation du tableau de bord et la note finale au cours des étudiants. Cette étude a ainsi contribué à l'avancement des connaissances, puisque très peu d'études ont traité de l'utilisation du tableau de bord des activités d'apprentissage par les étudiants dans des contextes éducatifs réels. Elle permet donc d'apporter un éclairage sur la fréquence et les périodes d'utilisation du tableau de bord par les étudiants selon plusieurs caractéristiques sociodémographiques. Cette étude a également permis d'approfondir le concept d'utilité du tableau de bord, en explorant les perceptions des étudiants sur le soutien que procure Ma réussite à l'autorégulation de l'apprentissage. Ces découvertes peuvent ainsi fournir de l'information aux gestionnaires de l'outil pour l'améliorer, notamment en ajoutant des fonctionnalités permettant de mieux soutenir l'autorégulation de l'apprentissage, et en informant davantage les parties prenantes de son fonctionnement. Finalement, il s'agit de la première étude à valider l'UTAUT en contexte d'utilisation du tableau de bord des activités d'apprentissage par les étudiants.Despite its growing popularity, online training in university context have to deal with issues related to perseverance and success, since dropout and failure rates are higher in this type of training than on campus training. In an effort to support student retention and success, Université Laval has developed a learning analytics dashboard that aims to make students aware of their participation in their courses, and to guide them toward help resources. This research focuses on the use of this dashboard (called Ma réussite) to support self-regulation of learning and the success of undergraduate online learning students. More specifically, it aims to document how and by whom the learning analytics dashboard is used as well as students' perceptions towards it. This thesis also aims to verify whether the Unified Theory of Acceptance and Use of Technology (UTAUT) can explain the use of the learning analytics dashboard, and whether its use is related to student success. To do this, we conducted a survey with 309 students in 15 different online courses across four faculties. We also obtained digital traces of student use of Ma réussite during the fall 2019 semester. Our results indicate that Ma réussite has low usage, and that the number of visits to Ma réussite is significantly higher for students taking their first semester in university or their first online course, and for those who are in contact with the tool for the first time. Using the self-regulation learning model (Zimmerman, 1998), we explored students' perceptions of the usefulness of the dashboard in supporting self-regulation of learning. Our results indicate that students agree that Ma réussite supports self-regulation of learning, including monitoring and regulation of effort. New entrants and students who are in contact with the tool for the first time perceive the dashboard more as a support for self-regulation. We also documented students' perceptions of the variables that predict use according to UTAUT: performance expectancy, effort expectancy, social influence, facilitating conditions, and intention to use. Our results indicate that Ma réussite is useful, easy to use and that the conditions facilitating its use are in place. However, the social influence to use it is perceived to be low. New entrants and students who are in contact with the tool for the first time perceive Ma réussite as more useful and intend more to use it in the future than other students. We also conducted partial least squares structural equation modeling to validate UTAUT, tailored to the study. Our results confirm hypotheses that performance expectancy, effort expectancy, social influence, and current use influence intention to use. However, our results indicate that facilitating conditions are not a determinant of intention to use. Our research model explains 62% of the variance in intention to use and 72% of the variance of performance expectancy, with self-regulation support as the only predictor variable. Finally, our results did not confirm a relationship between learning analytics dashboard use and student final grade. This study has contributed to the advancement of knowledge, since very few studies have examined the use of the learning analytics dashboard by students in real educational contexts. It therefore provides insight into the frequency and timing of student dashboard use across several socio-demographic characteristics. This study also further explored the concept of dashboard utility by exploring students' perceptions of the support that Ma réussite provides for self-regulation of learning. These findings can thus provide information for managers of the tool to improve it, including adding features to better support self-regulation of learning and further informing stakeholders about how it works. Finally, this is the first study to validate UTAUT in the context of student use of a learning analytics dashboard