Evidence-Based Assessment of Student Performance in Virtual Worlds

Virtual Worlds (VWs) are popular tools for teaching/learning in the twenty-first century classroom. The challenge remains however, to provide the means by which teachers could sustainably analyse and assess the performance of large groups of students in such environments. Unfortunately, external game features such as game scores and play duration have turned out to be unfair in some assessments. In this context, a case study was carried out in a foreign language course, illustrating how teachers could easily retrieve a number of performance indicators from VW-interaction logs and harness them to conduct a fine-grained analysis of students' performance, while facilitating at the same time valuable tools for their assessment. Objective performance indicators in a server database were made accessible using an end-user development programming language. This way, a range of data visualisation methods could be employed to contrast different assumptions regarding learner performance when playing a VW-based game, which was designed to help CEFR A1 level students to learn German. This way, factors such as randomisation of game tasks, which could negatively affect learner performance, were alleviated

On Data-driven systems analyzing, supporting and enhancing users’ interaction and experience

Tesis doctoral en inglés y resumen extendido en español[EN] The research areas of Human-Computer Interaction and Software Architectures have been traditionally treated separately, but in the literature, many authors made efforts to merge them to build better software systems. One of the common gaps between software engineering and usability is the lack of strategies to apply usability principles in the initial design of software architectures. Including these principles since the early phases of software design would help to avoid later architectural changes to include user experience requirements. The combination of both fields (software architectures and Human-Computer Interaction) would contribute to building better interactive software that should include the best from both the systems and user-centered designs. In that combination, the software architectures should enclose the fundamental structure and ideas of the system to offer the desired quality based on sound design decisions. Moreover, the information kept within a system is an opportunity to extract knowledge about the system itself, its components, the software included, the users or the interaction occurring inside. The knowledge gained from the information generated in a software environment can be used to improve the system itself, its software, the users’ experience, and the results. So, the combination of the areas of Knowledge Discovery and Human-Computer Interaction offers ideal conditions to address Human-Computer-Interaction-related challenges. The Human-Computer Interaction focuses on human intelligence, the Knowledge Discovery in computational intelligence, and the combination of both can raise the support of human intelligence with machine intelligence to discover new insights in a world crowded of data. This Ph.D. Thesis deals with these kinds of challenges: how approaches like data-driven software architectures (using Knowledge Discovery techniques) can help to improve the users' interaction and experience within an interactive system. Specifically, it deals with how to improve the human-computer interaction processes of different kind of stakeholders to improve different aspects such as the user experience or the easiness to accomplish a specific task. Several research actions and experiments support this investigation. These research actions included performing a systematic literature review and mapping of the literature that was aimed at finding how the software architectures in the literature have been used to support, analyze or enhance the human-computer interaction. Also, the actions included work on four different research scenarios that presented common challenges in the Human-Computer Interaction knowledge area. The case studies that fit into the scenarios selected were chosen based on the Human-Computer Interaction challenges they present, and on the authors’ accessibility to them. The four case studies were: an educational laboratory virtual world, a Massive Open Online Course and the social networks where the students discuss and learn, a system that includes very large web forms, and an environment where programmers develop code in the context of quantum computing. The development of the experiences involved the review of more than 2700 papers (only in the literature review phase), the analysis of the interaction of 6000 users in four different contexts or the analysis of 500,000 quantum computing programs. As outcomes from the experiences, some solutions are presented regarding the minimal software artifacts to include in software architectures, the behavior they should exhibit, the features desired in the extended software architecture, some analytic workflows and approaches to use, or the different kinds of feedback needed to reinforce the users’ interaction and experience. The results achieved led to the conclusion that, despite this is not a standard practice in the literature, the software environments should embrace Knowledge Discovery and data-driven principles to analyze and respond appropriately to the users’ needs and improve or support the interaction. To adopt Knowledge Discovery and data-driven principles, the software environments need to extend their software architectures to cover also the challenges related to Human-Computer Interaction. Finally, to tackle the current challenges related to the users’ interaction and experience and aiming to automate the software response to users’ actions, desires, and behaviors, the interactive systems should also include intelligent behaviors through embracing the Artificial Intelligence procedures and techniques

On data-driven systems analyzing, supporting and enhancing users’ interaction and experience

[EN]The research areas of Human-Computer Interaction and Software Architectures have been traditionally treated separately, but in the literature, many authors made efforts to merge them to build better software systems. One of the common gaps between software engineering and usability is the lack of strategies to apply usability principles in the initial design of software architectures. Including these principles since the early phases of software design would help to avoid later architectural changes to include user experience requirements. The combination of both fields (software architectures and Human-Computer Interaction) would contribute to building better interactive software that should include the best from both the systems and user-centered designs. In that combination, the software architectures should enclose the fundamental structure and ideas of the system to offer the desired quality based on sound design decisions. Moreover, the information kept within a system is an opportunity to extract knowledge about the system itself, its components, the software included, the users or the interaction occurring inside. The knowledge gained from the information generated in a software environment can be used to improve the system itself, its software, the users’ experience, and the results. So, the combination of the areas of Knowledge Discovery and Human-Computer Interaction offers ideal conditions to address Human-Computer-Interaction-related challenges. The Human-Computer Interaction focuses on human intelligence, the Knowledge Discovery in computational intelligence, and the combination of both can raise the support of human intelligence with machine intelligence to discover new insights in a world crowded of data. This Ph.D. Thesis deals with these kinds of challenges: how approaches like data-driven software architectures (using Knowledge Discovery techniques) can help to improve the users' interaction and experience within an interactive system. Specifically, it deals with how to improve the human-computer interaction processes of different kind of stakeholders to improve different aspects such as the user experience or the easiness to accomplish a specific task. Several research actions and experiments support this investigation. These research actions included performing a systematic literature review and mapping of the literature that was aimed at finding how the software architectures in the literature have been used to support, analyze or enhance the human-computer interaction. Also, the actions included work on four different research scenarios that presented common challenges in the Human- Computer Interaction knowledge area. The case studies that fit into the scenarios selected were chosen based on the Human-Computer Interaction challenges they present, and on the authors’ accessibility to them. The four case studies were: an educational laboratory virtual world, a Massive Open Online Course and the social networks where the students discuss and learn, a system that includes very large web forms, and an environment where programmers develop code in the context of quantum computing. The development of the experiences involved the review of more than 2700 papers (only in the literature review phase), the analysis of the interaction of 6000 users in four different contexts or the analysis of 500,000 quantum computing programs. As outcomes from the experiences, some solutions are presented regarding the minimal software artifacts to include in software architectures, the behavior they should exhibit, the features desired in the extended software architecture, some analytic workflows and approaches to use, or the different kinds of feedback needed to reinforce the users’ interaction and experience. The results achieved led to the conclusion that, despite this is not a standard practice in the literature, the software environments should embrace Knowledge Discovery and datadriven principles to analyze and respond appropriately to the users’ needs and improve or support the interaction. To adopt Knowledge Discovery and data-driven principles, the software environments need to extend their software architectures to cover also the challenges related to Human-Computer Interaction. Finally, to tackle the current challenges related to the users’ interaction and experience and aiming to automate the software response to users’ actions, desires, and behaviors, the interactive systems should also include intelligent behaviors through embracing the Artificial Intelligence procedures and techniques

Experimental Feedback on Prog&Play: A Serious Game for Programming Practice

International audienceThis paper presents an experimental feedback on a serious game dedicated to strengthening programming skills. This serious game, called Prog&Play, is built on an open source real-time strategy game. Its goal is to be compatible with different students, teachers and institutions. We based its evaluation on an iterative process that allows to implement the game and carry out experimentations in several contexts. Through this assessment, we define a framework which has been tested by third parties and we analyse both positive and negative points in order to improve the project. Evaluation is indeed beneficial and enables you to establish communication about the implemented practices

Evalsim: sistema para la extracción de indicadores de interacción en mundos virtuales

Este proyecto tiene como objetivo la creación de la herramienta EvalSim, que será usada como apoyo a la investigación docente. En la Universidad de Cádiz hay desplegada una plataforma de mundos virtuales que es usada para fomentar el aprendizaje autónomo de alumnos que estudian idiomas, en concreto nivel A1 de Alemán. Estos mundos virtuales generan gran cantidad de datos de interacción entre los jugadores, y entre los jugadores y el mundo virtual. EvalSim es un DSL (Lenguaje de Domino Específico) con una sintaxis muy sencilla, que se encarga de realizar procesos ETL (Extracción, Transformación y Carga) de todos los datos que generan los mundos virtuales, de forma que los investigadores obtengan indicadores que permitan analizar esa interacción y realizar estudios

Using Virtual Worlds to Identify Multidimensional Student Engagement in High School Foreign Language Learning Classrooms

Virtual world environments have evolved from object-oriented, text-based online games to complex three-dimensional immersive social spaces where the lines between reality and computer-generated begin to blur. Educators use virtual worlds to create engaging three-dimensional learning spaces for students, but the impact of virtual worlds in comparison to the traditional face-to-face counterpart has been uncertain in terms of multidimensional student engagement. Research has a need to determine the impact of virtual worlds on student engagement in comparison to the traditional face-to-face environment. The study examined the effects of virtual world and face-to-face learning environments on high school foreign language students\u27 emotional, cognitive, and behavioral engagement, as well as combined engagement. A two-way MANOVA was used to determine the effect of traditional face-to-face and virtual world learning environments on combined student engagement. A 2 x 2 analysis of covariance was used to determine the effect of traditional face-to-face and virtual world learning environments on emotional student engagement. A 2 x 2 analysis of covariance was also used to determine the effect of traditional face-to-face and virtual world learning environments on cognitive student engagement. A t-test was used to determine the effect of traditional face-to-face and virtual world learning environments on behavioral engagement. The study did not find evidence of overall, cognitive, emotional, or behavioral engagement difference between the two learning environments. The findings indicate the virtual world environment is similar to the traditional face-to-face environment in terms of student engagement. School administrators and teachers can benefit from this research when determining effective means of creating highly engaging learning environments for students. Virtual worlds can be a medium for engaging learning opportunities for students in face-to-face and virtual schools. Additional research in this area is recommended to determine the impact of virtual worlds with different student populations and subject areas

Framework for collaborative knowledge management in organizations

Nowadays organizations have been pushed to speed up the rate of industrial transformation to high value products and services. The capability to agilely respond to new market demands became a strategic pillar for innovation, and knowledge management could support organizations to achieve that goal. However, current knowledge management approaches tend to be over complex or too academic, with interfaces difficult to manage, even more if cooperative handling is required. Nevertheless, in an ideal framework, both tacit and explicit knowledge management should be addressed to achieve knowledge handling with precise and semantically meaningful definitions. Moreover, with the increase of Internet usage, the amount of available information explodes. It leads to the observed progress in the creation of mechanisms to retrieve useful knowledge from the huge existent amount of information sources. However, a same knowledge representation of a thing could mean differently to different people and applications. Contributing towards this direction, this thesis proposes a framework capable of gathering the knowledge held by domain experts and domain sources through a knowledge management system and transform it into explicit ontologies. This enables to build tools with advanced reasoning capacities with the aim to support enterprises decision-making processes. The author also intends to address the problem of knowledge transference within an among organizations. This will be done through a module (part of the proposed framework) for domain’s lexicon establishment which purpose is to represent and unify the understanding of the domain’s used semantic

A Serious Games Development Environment

Un ambiente per lo sviluppo di Serious Game