Teacher-Centered Dashboards Design Process

International audienceInterest in dashboards in schools has been growing in recent years as they have great potential in fostering data transparency and informing teachers' practices. However, research surrounding them is not unified and even less transparent, because of a lack of guidance in grounding their design as a process tailored to end-users needs. We present a process model for teacher-centered dashboards as a design and validation process with four mutually informed stages: (1) situate the domain space by framing teachers' routines, practices, and needs, (2) ideate the domain into multiple alternatives and prototypes, (3) develop, and (4) evaluate the dashboard. We drive recommendations within each stage to inform the process. We borrowed the foundations of the model from research in the HCI and InfoVis fields. We apply our model to five case studies from literature. We find that this model can provide a framework to scaffold dashboards' design process, mutually inform the underlying stages, and guide consolidating artifacts. We reflect on our work to provide design implications to point towards explainable dashboards design to best support teachers

Facteurs pour la conception et l'utilisation des tableaux de bord pour informer les pratiques enseignantes in situ

Les tableaux de bord ont un grand potentiel pour informer le travail, les décisions et les pratiques en éducation. Nous étudions la conception et l'utilisation des tableaux de bord pour informer les pratiques des enseignants lorsqu'ils aident les apprenants à utiliser les plates-formes d'apprentissage en ligne. Nous examinons spécifiquement ce que les enseignants "font" réellement avec ces technologies dans leur travail quotidien et à long terme. Nous soutenons que cette recherche a des ramifications sociales, pédagogiques et techniques. Pour comprendre les facteurs sociaux, nous avons besoin d'une enquête critique sur les pratiques pédagogiques et les défis techniques sous-tendus par les tableaux de bord des enseignants. D'autre part, pour concevoir de telles technologies, nous avons besoin de comprendre et considérer profondément les interactions sociales des utilisateurs.Nous avons étudié les tableaux de bord dans le contexte d'apprentissage de langues par vidéoconférence. Les enseignants sont confrontés à un défi majeur : le manque de conscience émotionnelle envers les apprenants en ligne en raison des interactions distantes et médiées par la technologie. Nous avons mené une étude de cas examinant une approche multimodale à la sensibilisation émotionnelle. Nous avons démontré cette approche en utilisant des flux audio et vidéo pour inférer des émotions, des émotions auto-rapportées par les apprenants, et les traces d'interaction contextuelle des enseignants/apprenants sur la plateforme. Nous avons analysé les données émotionnelles, audio et vidéo. Nous avons proposé une approche permettant de combiner les modèles d'émotions discrètes et dimensionnelles. La résultats soulignent la pertinence d'informations multimodales et contextuelles.Ensuite, sur la base de notre première étude, nous avons adopté un processus de conception itératif. Nous avons interviewé cinq enseignants et collaboré avec un responsable pédagogique, un chercheur en langues et deux enseignants. Nous avons mené deux évaluations formelles aboutissant à la conception d'Emodash. Nous avons mené une étude de terrain de deux mois avec cinq paires d'enseignants-apprenants, afin d'examiner comment Emodash sensibilise les enseignants aux émotions des apprenants en ligne et comment cela impact leurs bilans pédagogiques écrits aux apprenants. Les enseignants ont écrit des bilans significativement plus affectifs et formatifs, et moins sommatifs, ce qui suggère une grande prise de conscience des émotions des apprenants. De plus, les enseignants ont approprié Emodash comme un moyen d'évaluation et de réflexion sur leurs pratiques.Après, nous avons étudié les tableaux de bord dans le contexte d'apprentissage mixte : en ligne et en classe. Les enseignants sont confrontés à un défi majeur : le manque d'informations précises pour intervenir auprès des apprenants. Nous avons conçu Progdash sur la base d'entretiens avec sept enseignants et l'avons affiné grâce à un prototypage collaboratif. Nous avons intégré Progdash dans une plateforme d'apprentissage du français en ligne. Nous avons mené une étude de terrain de trois mois avec 29 enseignants pour examiner si Progdash fournit des indicateurs utiles aux enseignants, et comment il impact leurs pratiques. Les enseignants ont trouvé que Progdash facilitait : le suivi, les évaluations, la planification, le partage d'informations, le feedback et leçons. Enfin, nous avons mené une enquête réflexive en nous appuyant sur les résultats d'Emodash et de Progdash. Nous avons articulé les facteurs sociaux - suivi, sensibilisation et réflexion, les pratiques pédagogiques - planification, feedback et accompagnement, et les défis techniques qui interagissent avec la conception et l'utilisation des tableaux de bord des enseignants. Ces dimensions servent de cadre conceptuel pour les technologies de l'information et de macro-implications pour des tableaux de bord adaptés aux besoins et situations des enseignants et des apprenants.Dashboards have great potential in informing teachers' and learners' work, decisions, and practices. In this dissertation, we study the design and use of dashboards to inform teachers' practices when assisting learners in using online learning platforms. We specifically examine what teachers actually "do" with dashboards in their everyday work and in the long run. We argue that this research draws social, pedagogical, and technical ramifications. To understand the social factors, we need a critical inquiry into the pedagogical practices and the technical challenges underpinned by teachers' dashboards. On the other hand, to properly design such technologies, we need a profound understanding and consideration of the social interactions of users.First, we studied dashboards in the context of video-conferencing language learning. Teachers face one main challenge: lack of emotional awareness in online learning due to distant and technology-mediated interactions. We conducted a case study examining a multimodal approach of learners' emotions awareness. We demonstrated this approach by using audio and video streams when inferring emotions along with learners' self-reported emotions and teachers' and learners' contextual interaction traces on the platform. We analyzed emotional cues from the two modalities, audio, and video. We proposed an approach for combining discrete and dimensional emotion. The results highlighted the pertinence of rendering multimodal and contextual emotional awareness information for teachers in such settings.Second, building on our first study, we adopted an iterative design process in which we interviewed five teachers and collaborated with a pedagogical manager, a language researcher, and two teachers. We conducted two formal formative evaluations leading to the design of Emodash. We conducted a two-month field study with five teacher-learner pairs, to examine how Emodash supports teachers' awareness of learners' emotions online and how it impacts their feedback reports written to learners. The results showed that Emodash led teachers to write significantly more affective and formative feedback, and less summative feedback, suggesting a stronger awareness of learners' emotions. Also, the dashboard led teachers to reflect on the way they conduct lessons, using learners' emotions as a proxy to evaluate their conduct of teaching.Third, we studied dashboards in the context of remote and blended learning. Teachers face one main challenge: lack of actionable insights to engage in informed interventions. We designed Progdash based on interviews with seven teachers and refined it through collaborative prototyping. We integrated Progdash into an online French vocabulary, grammar, spelling learning platform. We conducted a three-month field study with 29 teachers to evaluate whether Progdash provides useful indicators to teachers about learners' progression, and how it informs teachers' practices to engage in informed interventions. The results showed that teachers found Progdash actionable to inform: monitoring, assessments, planning, information sharing, feedback, and lesson provision. Based on our findings, we discussed implications aimed at improving dashboards to bridge online and in-class learning as well as to foster teachers' and learners' dialog and reflection.Finally, we took a reflexive inquiry building upon the results of the studies of Emodash and Progdash. We articulated the social factors —monitoring, awareness, and reflection, the pedagogical practices —planning, feedback, and coaching, as well as the technical challenges interacting with the design and use of teachers' dashboards. We discussed practical assumptions for each dimension to inform the design and use of teachers' dashboards. Together, these dimensions serve as a wider conceptual umbrella for the design of information-driven technologies and macro-implications for dashboards fitting teachers' and learners' everyday situations

The Group Folding Effect: The Role of Collaborative Process Structuring and Social Interaction in Group Work

International audienceGroup work involves a myriad of complex processes encompassing social, perceptual, cognitive, and contextual factors. However, there is a lack of empirical research on computer-supported group work processes and their impact on outcomes at different stages of group work, especially when creativity and quality of outcomes are significant. Group work processes can interfere and hinder productivity, which we refer to as the "group folding effect. " We designed a three-stage process structuring to enhance group work productivity. In a field study, we examined how process structuring shapes productivity in two sub-studies: design and peer feedback, each with 40 participants (N = 40). The results revealed that process structuring significantly improved both the quantity and quality of productivity. Additionally, process structuring appeared to reduce inhibitory effects of group work, such as negative priming, fixation on familiar ideas, and social comparison. We discuss the implications of this research in supporting productive group work processes in collaborative tools and insights into a pattern of the group folding effect

EMODA: a tutor oriented multimodal and contextual emotional dashboard

International audienceLearners' emotional state has proven to be a key factor for successful learning. Visualizing learners' emotions during synchronous on-line learning activities can help tutors in creating and maintaining socio-affective relationships with their learners. However, few dashboards offer emotional information on the learning activity. The current study focuses on synchronous interactions via a videoconferencing tool dedicated to foreign language training. We collected data on learners' emotions in real conditions during ten sessions (five sessions for two learners). We propose to adopt and combine different models of emotions (discrete and dimensional) and to use heterogeneous APIs for measuring learners' emotions from different data sources (audio, video, self-reporting and interaction traces). Based on a thorough data analysis, we propose an approach to combine different cues to infer information on learners' emotional states. We finally present the EMODA dashboard, an affective multimodal and contextual visual analytics dashboard, which allows the tutor to monitor learners' emotions and better understand their evolution during the synchronous learning activity

EMODA: a tutor oriented multimodal and contextual emotional dashboard

International audienceLearners' emotional state has proven to be a key factor for successful learning. Visualizing learners' emotions during synchronous on-line learning activities can help tutors in creating and maintaining socio-affective relationships with their learners. However, few dashboards offer emotional information on the learning activity. The current study focuses on synchronous interactions via a videoconferencing tool dedicated to foreign language training. We collected data on learners' emotions in real conditions during ten sessions (five sessions for two learners). We propose to adopt and combine different models of emotions (discrete and dimensional) and to use heterogeneous APIs for measuring learners' emotions from different data sources (audio, video, self-reporting and interaction traces). Based on a thorough data analysis, we propose an approach to combine different cues to infer information on learners' emotional states. We finally present the EMODA dashboard, an affective multimodal and contextual visual analytics dashboard, which allows the tutor to monitor learners' emotions and better understand their evolution during the synchronous learning activity

EMODASH: A dashboard supporting retrospective awareness of emotions in online learning

International audienceWe present EMODASH, an interactive dashboard supporting tutors' retrospective awareness of learners' emotions in a video-conferencing learning environment. Socio-affective relationships play an important role in learning processes and learning outcomes, but they are harder to develop in online-learning. This can be explained by a lack of emotion awareness due to the asynchronous interactions, technical challenges, and tutors' focus on properly conducting the learning activity and gearing towards pedagogical outcomes. We conducted an eight-week long field study with five professional tutors on how they used EMODASH while writing feedback to learners after language learning sessions. We found that EMODASH led tutors who were already sensitive to learners' emotions to incorporate more affective elements in their reports, suggesting a stronger awareness of learners' emotions. Tutors also wrote more formative and less summative feedback. Furthermore, our results suggest that glanceable visualizations of learners' emotions may be preferred and sufficient to foster tutors' awareness of learners' emotions. Finally, the dashboard led tutors to reflect on the way they conduct their lessons, using learners' positive emotions as a proxy evaluation of their teaching

PROGDASH: Lessons Learned From a Learning Dashboard in-the-wild

International audienceTeachers mixing in-class and remote online learning activities face numerous challenges in monitoring learners' online activity and progress, especially when preparing in-class interventions. We present a design study of PROGDASH, a dashboard enabling teachers to monitor learners' activity on an online grammar and spelling learning platform. We designed PROGDASH based on interviews with seven teachers to understand their needs, and we collaboratively iterated on design prototypes. We conducted a three-month field deployment with 17 teachers, collecting logs, diaries, and interview data to evaluate how PROGDASH informed their practices. Our findings extend prior work on teachers' practices using dashboards: teachers found PROGDASH actionable to devise informed pedagogical practices: monitoring, conducting assessments, planning interventions, sharing in-class, providing debriefing and feedback. Based on our findings, we suggest directions on how dashboards could effectively support teachers in schools using online learning platforms in authentic settings