Generative AI chatbot in PyramidApp: students’ behaviors and design principles

Abstract

Comunicació presentada a 30th International Conference CollabTech 2024, celebrada a Barcelona del 11 al 14 de setembre de 2024.Generative Artificial Intelligence (GenAI) offers new opportunities to implement useful features within Computer Supported Collaborative Learning (CSCL) environments. Despite these growing prospects, there is still limited research concerning the application of GenAI in learning environments. This work in progress aims to evaluate the mediation of a masked GenAI chatbot in the setting of the CSCL web application PyramidApp. A quasi-experimental within-subjects study was designed to assess the effect of GenAI chatbot intervention within the environment of PyramidApp. In the setting of 9 online activities, we evaluated the effect of the GenAI chatbot activity in 105 conversational chat rooms. The findings revealed that the GenAI chatbot provides useful feedback as students rate the chatbot’s answers higher than their peers’ answers (MChatbot = 4.11, MStudents = 3.91). The presence of the chatbot has an effect on group communication with the length of messages increased in chat rooms where the chatbot was present. Moreover, chatbot behavior to rate the students’ answers was correlated with the students’ behavior. The present study offers valuable insights into the optimal strategies for integrating a GenAI Large Language Model into educational tools and computer supported learning.This work was partially funded by the National Research Agency of the Spanish Ministry (PID2020-112584RB-C33, CEX2021–001195-M) and the Department of Research and Universities of the Government of Catalonia (SGR 00930). The work of D. Hernández-Leo (Serra Hunter) was supported by ICREA under the ICREA Academia program