Conceptual framework for process-oriented feedback through Learning Analytics Dashboards

The number of students enrolled in online higher education courses is increasing, and as a result, more data on their learning process is being generated. By exploring this student behavior data through learning analytics, both student and teacher can be provided with process-oriented feedback in the form of dashboards. However, little is known about the typology of relevant feedback in the dashboard to different learning objectives, students and teachers. Although most dashboards and the feedback they provide are based solely on student performance indicators, research shows that such feedback is not sufficient. This article attempts to define a conceptual model that visualizes the relationships between the design of a Learning Analytics Dashboard (LAD) and the concepts of learning science in order to provide process-oriented feedback that supports the regulation of learning. The aim of the work is not to propose a specific design of the LAD to provide feedback, but rather a conceptual framework for the choice of concepts for that design, and therefore to help understand future data needs as a basis for the educational feedback of the dashboards. As a conclusion of our research, we can say that having LADs adapted to any profile (student, teacher, etc.) can improve decision-making processes by showing each user the information that interests them most in the way that best enables them to understand it

PEMBANGUNAN DASHBOARD PERFORMA KELAS BAGI DOSEN UNIVERSITAS ATMA JAYA YOGYAKARTA BERBASIS WEBSITE

Dosen memiliki banyak tugas dan tanggung jawab yang perlu diemban pada pekerjaan sehari-hari. Selain harus melakukan kegiatan belajar-mengajar dengan mahasiswa, dosen juga perlu mengerjakan hal lain seperti mengadakan penelitian. Hal ini tentu membebani dosen terutama dalam hal keterbatasan waktu. Dosen memiliki waktu yang sangat terbatas untuk bisa memahami kondisi performa kelasnya, yang berakibat sulitnya menemukan metode belajar yang cocok pada masing-masing kelas. Ditambah lagi metode penilaian pada kurikulum baru yang memfokuskan penilaian pada aspek yang lebih mendetail seperti capaian mata kuliah, sehingga menambah waktu yang dibutuhkan untuk melakukan penilaian. Untuk menyelesaikan masalah tersebut, dibentuklah sistem dashboard performa kelas yang terdiri dari dua website dengan tujuan melakukan modifikasi data nilai dan menampilkan hasil perhitungan nilai. Website modifikasi data dibuat dengan teknologi Vue.js dan Laravel, sedangkan untuk website tampilan nilai digunakan Python dengan library utama Streamlit. Fitur yang dibuat pada sistem disesuaikan dengan kebutuhan dosen terkait dari hasil wawancara sebelumnya. Hasil dari penelitian yang dilakukan adalah sistem dashboard performa kelas yang sesuai dengan kebutuhan dosen di universitas Atma Jaya Yogyakarta. Sistem ini juga sudah diujikan kepada beberapa dosen untuk memastikan tercapainya tujuan utama dalam membantu dosen menilai dan memberikan insight terkait performa nilai kelas

Improving the Learning Process in the Higher Education Through the Use of a Predictive Tool (Dashboard)

Improving the learning process in the high education through the use of a predictive tool (dashboard). De Jesús Liriano, Rubėn and Sevillano, María C., 2018: A Florida Distance Learning Association Journal (FDLA) Paper to be published in a special section of the US Distance Learning Association Professional Journal Distance Learning. Interaction/Predictive Tools/Dashboard/Evaluation Learning Process/Automated Process/eLearning Environment/Learning Environment/Learning Management System (LMS)/Cloud Computing/ Learning Performance, and Learning Analytics. The purpose of this study was the initial step in providing a clear understanding of the use of a predictive tool (dashboard) to improving the learning process in the higher education field. Through further applications and research, the predictive tool must be refined to increase accuracy, and provide expeditious, reliable feedback about the students’ progress in an online environment. This information will benefit e-learning facilitators in the following ways: 1) knowing their students and, therefore, identifying how best to deliver content to them; 2) applying the right predictive tool to capitalizing on the new experience and new opportunity; 3) taking value-added approaches to student engagement; 4) bridging connections in a precise manner between enriched content and e-learning; and 5) transforming e-learning into digital experience

Understanding learning at a glance: An overview of learning dashboard studies

Research on learning dashboards aims to identify what data is meaningful to different stakeholders in education, and how data can be presented to support sense-making processes. This paper summarizes the main outcomes of a systematic literature review on learning dashboards, in the fields of Learning Analytics and Educational Data Mining. The query was run in five main academic databases and enriched with papers coming from GScholar, resulting in 346 papers out of which 55 were included in the final analysis. Our review distinguishes different kinds of research studies as well as different aspects of learning dashboards and their maturity in terms of evaluation. As the research field is still relatively young, many of the studies are exploratory and proof-of-concept. Among the main open issues and future lines of work in the area of learning dashboards, we identify the need for longitudinal research in authentic settings, as well as studies that systematically compare different dashboard design options

The Virtual Tutor: Combining Conversational Agents with Learning Analytics to support Formative Assessment in Online Collaborative Learning

The objective of this design science research project is to combine Learning Analytics data with a conversational agent communication interface, the Virtual Tutor, which is able to support formative assessment for educators and learners in online collaborative learning (OCL) environments. The main benefit for educators is providing user-adaptable Learning Analytics data requests to fit the information needs for formative assessment. Learners receive semi-automated feedback on their platform activity in form of reports, which shall trigger self-reflection processes. By extracting requirements from the potential users and deriving design principles, a conversational agent is implemented and evaluated in an online collaborative learning course. The results indicate that the Virtual Tutor reduces the task load of educators, supports formative assessment and gives scaffolded guidance to the learners by reflecting their performance, thus triggering self-reflection processes. This research provides a first step towards data supported (semi-)automated feedback systems for formative assessment in OCL courses

Créer un tableau de bord permettant de répondre aux besoins des résidents d’un programme de formation fondé sur les compétences : projet de recherche basé sur la conception

Background: Canadian specialty programs are implementing Competence By Design, a competency-based medical education (CBME) program which requires frequent assessments of entrustable professional activities. To be used for learning, the large amount of assessment data needs to be interpreted by residents, but little work has been done to determine how visualizing and interacting with this data can be supported. Within the University of Saskatchewan emergency medicine residency program, we sought to determine how our residents’ CBME assessment data should be presented to support their learning and to develop a dashboard that meets our residents’ needs. Methods: We utilized a design-based research process to identify and address resident needs surrounding the presentation of their assessment data. Data was collected within the emergency medicine residency program at the University of Saskatchewan via four resident focus groups held over 10 months. Focus group discussions were analyzed using a grounded theory approach to identify resident needs. This guided the development of a dashboard which contained elements (data, analytics, and visualizations) that support their interpretation of the data. The identified needs are described using quotes from the focus groups as well as visualizations of the dashboard elements. Results: Resident needs were classified under three themes: (1) Provide guidance through the assessment program, (2) Present workplace-based assessment data, and (3) Present other assessment data. Seventeen dashboard elements were designed to address these needs. Conclusions: Our design-based research process identified resident needs and developed dashboard elements to meet them. This work will inform the creation and evolution of CBME assessment dashboards designed to support resident learning.Contexte : Les programmes canadiens de spécialité sont à implanter la compétence par conception (CPC), un programme d’éducation médicale par compétences qui nécessite des évaluations fréquentes des activités professionnelles confiables. Pour servir aux fins d’apprentissage, la grande quantité de données d’évaluation doit être interprétée par les résidents, mais peu de travaux ont été réalisés pour déterminer comment la visualisation et l’interaction avec ces données peuvent être soutenues. Dans le cadre du programme de résidence en médecine d’urgence de l’Université de Saskatchewan, nous avons cherché à déterminer comment les données d’évaluation de la CPC de nos résidents devraient être présentées pour soutenir leur apprentissage et pour développer un tableau de bord qui réponde aux besoins de nos résidents. Méthodologie : Nous avons utilisé un processus de recherche orientée par la conception pour cerner les besoins des résidents en lien avec la présentation de leurs données d’évaluation. Les données ont été recueillies au cours du programme de résidence en médecine d’urgence de l’Université de Saskatchewan grâce à quatre groupes de discussion de résidents qui se sont tenus sur une période de 10 mois. Les groupes de discussion ont été analysés en utilisant l’approche de la théorisation ancrée (Grounded Theory) pour cerner les besoins des résidents, pour guider le développement d’un tableau de bord contenant des éléments (données, analyses et visualisations) qui soutiennent leur interprétation de leurs propres données. Les besoins identifiés sont décrits à l’aide de citations des groupes de discussion ainsi que de visualisations des éléments du tableau de bord. Résultats : Les besoins des résidents ont été classés sous trois thèmes : 1. être guidés quant au programme d'évaluation, 2. présenter des données d’évaluation en milieu de travail, et 3. présenter d’autres données d’évaluation. Dix-sept éléments du tableau de bord ont été conçus pour répondre à ces besoins. Conclusions : Notre méthode de recherche orientée par conception a permis de cerner les besoins des résidents et d’élaborer les éléments d’un tableau de bord pour y répondre. Ce travail servira de base à la création et à l’évolution des tableaux de bord d’évaluation en CPC conçus pour soutenir l’apprentissage des résidents

Revisão sistemática sobre painéis de visualização de dados educacionais

Um painel de visualizações de dados educacionais se utiliza dos dados gerados nas plataformas online de aprendizagem e os organiza em indicadores gráficos, como dados de acesso, permanência, desempenho ou outros sinalizadores que possam informar aos estudantes, professores e gestores sobre o processo de aprendizagem, evolução e desempenho. Tais painéis podem ser ferramentas importantes em contextos de educação à distância, devido à grande quantidade de estudantes e à pouca interação entre a plataforma, os professores e os estudantes. A fim de explorar estes recursos, neste artigo foi feita uma revisão sistemática sobre este tema, com o objetivo de identificar tendências e lacunas em relação ao emprego destas ferramentas em plataformas de ensino online. A ausência de pesquisas sobre a utilização de painéis específicos para cursos massivos abertos online (Massive Open Online Course – MOOCs) e a comparação entre diferentes cursos foram constatados neste estudo

VLA Dashboard: Um Mecanismo para Visualização do Desempenho de Estudantes de Matemática no Ensino Médio

Este artigo descreve as contribuições do uso de um mecanismo VLA Dashboard para visualização do desempenho de estudantes do Ensino Médio em avaliações de Matemática Básica aplicadas no Moodle. Diante disso, empregou-se uma pesquisa qualitativa do tipo Estudo de Caso. Em seguida, a coleta foi aplicada em três etapas: pré-Teste, oficina de capacitação e aplicação de avaliação diagnóstica pelos professores. Também, foi usado o Modelo de Aceitação de Tecnologia para os questionários. Os resultados apontam que o uso das técnicas de Visualização da Informação, learning analytics e abordagem multiagente contribuem para melhorar os diagnósticos das atividades avaliativas do professor