Learning to Prompt in the Classroom to Understand AI Limits: A pilot study

Artificial intelligence's progress holds great promise in assisting society in addressing pressing societal issues. In particular Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems allowing them to process an unprecedented amount of unstructured data. The consequent hype has also backfired, raising negative sentiment even after novel AI methods' surprising contributions. One of the causes, but also an important issue per se, is the rising and misleading feeling of being able to access and process any form of knowledge to solve problems in any domain with no effort or previous expertise in AI or problem domain, disregarding current LLMs limits, such as hallucinations and reasoning limits. Acknowledging AI fallibility is crucial to address the impact of dogmatic overconfidence in possibly erroneous suggestions generated by LLMs. At the same time, it can reduce fear and other negative attitudes toward AI. AI literacy interventions are necessary that allow the public to understand such LLM limits and learn how to use them in a more effective manner, i.e. learning to "prompt". With this aim, a pilot educational intervention was performed in a high school with 30 students. It involved (i) presenting high-level concepts about intelligence, AI, and LLM, (ii) an initial naive practice with ChatGPT in a non-trivial task, and finally (iii) applying currently-accepted prompting strategies. Encouraging preliminary results have been collected such as students reporting a) high appreciation of the activity, b) improved quality of the interaction with the LLM during the educational activity, c) decreased negative sentiments toward AI, d) increased understanding of limitations and specifically We aim to study factors that impact AI acceptance and to refine and repeat this activity in more controlled settings.Comment: Submitted to AIXIA 2023 22nd International Conference of the Italian Association for Artificial Intelligence 6 - 9 Nov, 2023, Rome, Ital

Challenging Social Media Threats using Collective Well-being Aware Recommendation Algorithms and an Educational Virtual Companion

Social media (SM) have become an integral part of our lives, expanding our inter-linking capabilities to new levels. There is plenty to be said about their positive effects. On the other hand however, some serious negative implications of SM have repeatedly been highlighted in recent years, pointing at various SM threats for society, and its teenagers in particular: from common issues (e.g. digital addiction and polarization) and manipulative influences of algorithms to teenager-specific issues (e.g. body stereotyping). The full impact of current SM platform design -- both at an individual and societal level -- asks for a comprehensive evaluation and conceptual improvement. We extend measures of Collective Well-Being (CWB) to SM communities. As users' relationships and interactions are a central component of CWB, education is crucial to improve CWB. We thus propose a framework based on an adaptive "social media virtual companion" for educating and supporting the entire students' community to interact with SM. The virtual companion will be powered by a Recommender System (CWB-RS) that will optimize a CWB metric instead of engagement or platform profit, which currently largely drives recommender systems thereby disregarding any societal collateral effect. CWB-RS will optimize CWB both in the short term, by balancing the level of SM threat the students are exposed to, as well as in the long term, by adopting an Intelligent Tutor System role and enabling adaptive and personalized sequencing of playful learning activities. This framework offers an initial step on understanding how to design SM systems and embedded educational interventions that favor a more healthy and positive society

Computer-supported scripting mechanisms to support adolescents' development of social media self-protection skills

In recent years, there's been increasing concern over potential negative experiences for adolescents on social media. Despite efforts in social media literacy education, traditional approaches often neglect crucial affective structures necessary for well-being and resilience in this context. This doctoral thesis introduces the concept of Social Media Self-Protection Skills (SMSPS), aiming to connect social media literacy with emotional competencies. The thesis explores various learning strategies to acquire SMSPS and proposes an authentic learning environment equipped with narratives, self-reflection and computer-supported collaborative learning activities. Beyond the platform's development, this thesis addresses limitations in current methodologies by proposing scripting mechanisms to overcome them. Using a design-based research methodology, this work iteratively refines an educational social media platform through user feedback, design refinements, and real-world evaluation studies. Results include design artifacts, societal outputs, and design principles informing future educational social media platforms for fostering SMSPS.En recientes años, ha aumentado la inquietud por las experiencias negativas de los adolescentes en redes sociales. A pesar de la educación en alfabetización en redes sociales, los métodos tradicionales descuidan estructuras afectivas esenciales para el bienestar y la resistencia en este entorno.Esta tesis doctoral introduce el concepto de Habilidades de Autoprotección en Redes Sociales (SMSPS), con el objetivo de conectar la alfabetización en redes sociales con las competencias emocionales. La tesis explora diversas estrategias de aprendizaje para adquirir SMSPS y propone un entorno de aprendizaje auténtico equipado con narrativas, autorreflexión y actividades de aprendizaje colaborativas. Con una metodología de investigación basada en el diseño, este trabajo perfecciona una plataforma educativa de redes sociales mediante retroalimentación de usuarios y evaluaciones en entornos reales. Los resultados incluyen artefactos de diseño, impactos sociales y principios para desarrollar futuras plataformas educativas de redes sociales que fomenten las SMSPS.Programa de Doctorat en Tecnologies de la Informació i les Comunicacion

Gender-based learning and behavioural differences in an educational social media platform

Data de publicació electrònica: 19 de desembre del 2023Background:Gender differences in the use of educational environments and socialmedia platforms have been a topic of interest in research. Several studies have exam-ined the disparities between female and male participants in terms of participation,engagement, and motivations in either educational or social media platforms. On thisend, this study proceeds to explore gender disparities in an innovative educationalenvironment that sees the incorporation of an educational component within a simu-lated social media environment.Objectives:Driven by a dual objective, this study aims to first investigate the impactof gender on skills development in an educational social media platform, and second,to explore the behavioural differences between female and male adolescents socialinteractions within the platform.Methods:The study involved 142 high school students in Barcelona and utilised anad-hoc questionnaire to assess students' skills. To explore students' behaviour withinthe platform, actions such as likes, comments, publishing, visiting profiles, and view-ing images, were recorded using the Experience API.Results and Conclusions:Our study's findings indicate that both female and male stu-dents benefited from the intervention implemented within the educational social mediaplatform. Moreover, the results revealed distinct interaction patterns between femaleand male participants. These findings provide valuable insights into how adolescents ofdifferent genders interact within an educational social media platform. Additionally, thedesign principles outlined in this study can inform the development of future educationalsocial media interventions that effectively engage the preferences of both genders.This work has been partially funded by the Volkswagen Foundation(COURAGE project, no. 95567), by the Ministerio de Ciencia e Innovaci on:MICIN/AEI/10.13039/501100011033 in PID2020-112584RB-C33 andMDM-2015-0502. DHL (Serra Hunter) also acknowledges the invaluablesupport provided by ICREA under the ICREA Academia program and theDepartment of Research and Universities of the Goverment of Catalonia(SGR 00930)

Towards socially shared regulation within CSCL scripts: mirroring group participation in PyramidApp

Comunicació presentada a: LASI Spain celebrat de manera híbrida del 7 al 9 de juliol de 2021 a Barcelona, Espanya.Computer-Supported Collaborative Learning (CSCL) has been shown to enhance learning by promoting peer interactions. With the support of collaborative learning flow patterns (CLFP), the collaboration process within the socalled CSCL macro-scripts can be further structured and constrained. However, social dynamics within tasks, even within scripts, have the potential to hinder the collaborative element as students might not participate sufficiently in conversations. To determine if participation in discussions between students was balanced within a CSCL script we analysed data from an implementation of the Pyramid CLFP. The data showed that student participation varied across different groups, with some groups and students not initiating conversation between them. Socially shared regulation is a social form of regulation where students regulate their learning as a group rather than as individuals through the means of discussion, negotiation, and perspective taking. As research has shown that for collaborative learning to be successful students need to engage in fruitful discussions and support each other to regulate their learning, we propose the implementation of a social awareness feature mirroring group participation through learning analytics in a CLFP tool (PyramidApp) to promote socially shared regulated learning.This work has been partially funded by the Volkswagen Foundation in the framework of the project COURAGE (no. 95567) and by FEDER, the National Research Agency of the Spanish Ministry, TIN2017-85179-C3-3-R and MDM-2015-0502. D. Hernández-Leo also acknowledges the support of ICREA under the ICREA Academia programme

Uso de redes sociales en adolescentes catalanes. Hábitos y competencias mediáticas

Comunicació presentada a VII Congresso Ibero-Americano de Comunicação 2022: IBERCOM 2022, conjuntament amb XVII Congresso Ibero-Americano de Comunicação, celebrada del 26 al 28 d'octubre de 2022 a Oporto, Portugal.El cambio de paradigma comunicacional hacia una cultura que se virtualiza sitúa a las redes sociales como el principal espacio de socialización para jóvenes y adultos. Datos recientes señalan que los adolescentes en España emplean una media de 5 horas navegando en internet, de las cuales el mayor tiempo consumido es en redes sociales. Esta tendencia que no es lejana a la de otros países industrializados genera preocupación no sólo por la prolongada exposición y sus efectos en el bienestar de los menores sino también por los riesgos que conlleva el uso de redes sociales. En este trabajo se analizan los hábitos de consumo y las competencias en redes sociales de adolescentes escolarizados en la Comunidad Autónoma de Cataluña en el 2021. El estudio transversal de tipo descriptivo analiza los datos de 1224 estudiantes de Barcelona y otros municipios colindantes. Como parte del estudio, se analizaron cuatro perfiles sociodemográficos (grupo etario, sexo, localidad escolar y tipo de centro). El estudio detalla un consumo generalizado de redes en los adolescentes, con diferencias notables respecto al grupo etario y sexo. A partir de los resultados del estudio se esbozan algunas recomendaciones para la actuación de la educación mediática con jóvenes en Cataluña

Students’ generated text quality in a narrative-centered learning environment: effects of pre-collaboration, individual, and chat-interface submissions

Comunicació presentada a 29th international conference, CollabTech 2023, celebrada del 29 d'agost al 1 de setembre de 2023 a Osaka, Japó.Narrative-centered Learning Environments (NcLE) offer a powerful tool for enhancing students’ learning through interactive experiences. The integration of open-ended questions in NcLEs encourages students to express their thoughts, opinions, and ideas without any restrictions. While different submission formats are implemented in NcLE, little research has argued the importance of assessing their potential effects. This study, therefore, investigates how the text quality in writing varies based on different submission formats for open-ended submissions in a NcLE. We proceed to analyze three types of submission formats within a narrative scripts platform: a) formal individual, b) pre-collaboration, and c) a chatbot interface. For this study, data was collected in a randomized controlled trial that involved 311 secondary school students participating in a social media literacy workshop. The quality of the submissions was assessed using an automated text cohesion and coherence analysis tool (Coh-Metrix). Our results show that having formal submission pages for open-ended questions has an effect in students’ efforts to create submissions with higher writing quality, while submissions via chatbot interfaces tend to be brief and conversational in nature. The results presented in this study have practical implications for learning technology developers and educators, as they can guide decisions related to the creation, and integration of elements to promote writing quality in online learning environments.This work has been partially funded by the Volkswagen Foundation (COURAGE project, no. 95567). TIDE-UPF also acknowledges the support by FEDER, the National Research Agency of the Spanish Ministry, PID2020-112584RB-C33/MICIN/AEI/10.13039/501100011033, MDM-2015-0502 and by ICREA under the ICREA Academia programme (D. Hernández-Leo, Serra Hunter) and the Department of Research and Universities of the Government of Catalonia (SGR 00930)

The Challenge of gathering self-reported moods: Cases using a classroom orchestration tool

Comunicació presentada a: ICALT 2020 International Conference on Advanced Learning Technologies and Technology-enhanced Learning, organitzada a Tartu, Estònia, del 6 al 9 de juliol de 2020Self-reports of affective states are increasingly being collected in educational settings. However, individual definitions and usage of emotion and mood terms are often subjective despite objective definitions becoming more widely accepted. We explore whether the variation among individual learners in how mood terms are defined presents an obstacle to using self-reported mood data for group comparison studies. Following a design-based research methodology, we ran two case studies to explore the use of ClassMood App in a multimodal learning study and the validity of the self-reported mood data it collected. During the first case, 24 primary school students experienced difficulty understanding the words used to describe the moods in ClassMood App. In the second case, involving 77 university students, we explored whether the misunderstanding of mood words persisted with older students. Participants were asked to rate their familiarity with and match definitions to a set of 8 mood words. We found that levels of familiarity with the mood words varied greatly and 17.9% of all definition matching attempts were incorrect. The results suggest that the variation in subjective definitions of mood terms is likely to affect the validity of the data collected by the ClassMood App for group comparison studies.This work has been partially funded by the European Regional Development Fund, Erasmus+, and the National Research Agency of the Spanish Ministry of Science, Innovation and Universities, under project grants TIN2017-85179-C3-3-R, 2018-1-ESOI-KA20I-050646, and MDM-2015-0502. D. Hernández-Leo acknowledges the support by ICREA under the ICREA Academia programme

Intrinsic motivation for social media literacy, a look into the narrative scripts

Comunicació presentada a 4th International Conference (HELMeTO 2022), celebrada del 21 al 23 de setembre de 2022 a Palerm, Itàlia.Social media embodies the idyllic representation of the information society providing users opportunities to connect and communicate. Likewise, it can represent humanity’s worst nightmare with a plethora of negative phenomena. Different forms of media literacy interventions have been designed as preventative tools to counteract the negative phenomena of social media. One such approach, the narrative scripts, sees the integration of a social media intervention within a controlled social media platform. With an importance to design engaging educational tools to get learners attention without limiting the educational deliverables, the narrative scripts approach sees the implementation of gamifying elements to help deliver the learning material and motivate learners to participate. To assess the effectiveness of the proposed motivational properties of the narrative scripts, a descriptive exploratory study was designed. In total 124 adolescents participated in a social media literacy workshop powered by the narrative scripts. During the workshop students completed missions assigned by a virtual learning companion and participated in collaborative learning activities. To measure the motivation levels of the students, the Intrinsic Motivation Inventory was applied. Results demonstrate that adolescents found the narrative scripts an interesting and enjoyable activity. Qualitative analysis supported the findings with adolescents describing the activities as dynamic and entertaining. The results from this study provide insights towards the motivational aspects of the narrative scripts as a social media intervention tool.This work has been partially funded by the National Research Agency of the Spanish Ministry (PID2020-112584RB-C33/MICIN/AEI/10.13039/501100011033, MDM2015–0502) and the Volkswagen Stiftung (Courage, ref. 95 566). D. Hernández-Leo (Serra Húnter) acknowledges the support by ICREA under the ICREA Academia programme

Narrative scripts embedded in social media towards empowering digital and self-protection skills

Social media has become an important part of adolescents' lives, with an increasing number of teenagers spending a great part of their time creating, sharing, and socializing with online content. Although the popularity of social media keeps growing, different studies identified threats and dangers that exist in such networks. From harmful content to negative behaviors, users can fall victim to negative social media phenomena that can affect their mental health and wellbeing. Several media literacy initiatives have been designed to promote social media awareness amongst the youth using traditional approaches to teaching about social media risks and threats. However, these approaches are limited in enabling deep reflection about the dangers behind their social media interactions and empowering their empathy, perspective-taking, critical thinking, digital and self-protection skills. This demo paper introduces a perspective in this context proposing the integration of educational opportunities within social media. The proposed approach is designed as a social media simulated learning platform where embedded learning activities follow a novel “narrative scripting” approach, in which Computer Supported Collaborative Learning script mechanisms are combined with counter-narratives strategies.This work has been partially funded by the Volkswagen Foundation (COURAGE project, no. 95567). TIDE-UPF also acknowledges he support by FEDER, the National Research Agency of the Spanish Ministry, TIN2017-85179-C3-3-R and by ICREA under the ICREA Academia programme (D. Hernández-Leo, Serra Hunter)