Design and Evaluation of a Collaborative Educational Game: BECO Games

This paper describes the design and validation of a game based on a platform for easy deployment of collaborative educational games, named BECO Games platform. As an example of its potential, a learning experience for an Economics subject was created through a collaborative game to understand the concept of common goods. The effectiveness of the game was tested by comparing the performance of Bachelor students who used the platform and those who did not (137 students vs. 92 students). In addition, it was controlled that in previous years when students played the game through forums and an Excel sheet, these differences did not exist. Results indicate that the performance differences between students who participated in the online game and those who did not were greater than in previous years. In addition, a satisfaction survey was delivered to the students to understand their impressions better. This survey assessed student opinion about the platform, about the educational experience, and about their behavior during the game

An Operational Framework for Evaluating the Performance of Learning Record Stores

International audienceNowadays, Learning Record Stores (LRS) are increasingly used within digital learning systems to store learning experiences. Multiple LRS software have made their appearance in the market. These systems provide the same basic functional features including receiving, storing and retrieving learning records. Further, some of them may offer varying features like visualization functions and interfacing with various external systems. However, the non-functional requirements such as scalability, response time and throughput may differ from one LRS to another. Thus, for a specific organization, choosing the appropriate LRS is of high importance, since adopting a non-optimized one in terms of non-functional requirements may lead to a loss of money, time and effort. In this paper, we focus on the performance aspect and we introduce an operational framework for analyzing the performance behaviour of LRS under a set of test scenarios. Moreover, the use of our framework provides the user with the possibility to choose the suitable strategy for sending storing requests to optimize their processing while taking into account the underlying infrastructure. A set of metrics are used to provide performance measurements at the end of each test. To validate our framework, we studied and analyzed the performances of two open source LRS including Learning Locker and Trax

Oppimisanalytiikan käynnistäminen, Tapaus: Aalto Online Learning

The digital transformation of learning brings forth data having unprecedented granularity and coverage of learning activity. The research area of Learning Analytics (LA) uses this data to understand and improve learning. The practice of LA is a cyclic process where learning data is collected from different sources and analytics is developed according to stakeholder objectives. Finally, current results are delivered that lead into action which improves learning and produces new data. The goal of this thesis is to bootstrap LA in multiple courses that implement different weekly online learning activities. The term bootstrap underlines the aim to support continuity, further development, and expansion of LA. The research questions were: what learning data the courses currently instrument, and what LA objectives the course staff find most important. This thesis conducts software engineering to construct an LA solution for the research case. Requirements are defined via examination of the case and interviews of the course staff. The developed solution enables real time access to learning data and possibility to integrate data from both Moodle and A-plus learning environments for joined analysis. Novel interactive visualizations are developed according to the user requirements. The work in bootstrapping LA at course level lead to two general findings. First, the integration of learning data from multitude of sources is a common challenge that requires design. Second, teachers' initial LA objectives include aims to monitor expected progress, improve allocation of learning material, identify problematic areas in learning material, and improve interaction with learners.Opetuksen digitaalinen murros synnyttaää ennennäkemättömän tarkkaa ja kattavaa tietoa oppimisaktiviteeteista. Oppimisanalytiikan (OA) tutkimusalue käyttää tätä aineistoa oppimisen ymmärtämiseen ja parantamiseen. OA:n soveltaminen käytäntöön on toistuva prosessi, jossa oppimisaineistoa kerätään erilaisista lähteistä ja analytiikkaa kehitetään omistajiensa tavoitteiden mukaisesti. Lopuksi tuotetaan ajantasaisia tuloksia, jotka johtavat toimintaan, joka parantaa oppimista ja tuottaa uutta aineistoa. Tämän diplomityön tavoitteena on käynnistää OA usealla kurssilla, jotka toteuttavat erilaisia viikoittaisia verkko-oppimisen ratkaisuja. Käynnistäminen pyrkii elinvoimaiseen, kehittyvään ja laajenevaan analytiikkaan. Tutkimuskysymykset olivat, mitä dataa kurssit tällä hetkellä keräävät ja mitkä OA–tavoitteet ovat kurssihenkilökunnalle tärkeimpiä. Työssä rakennetaan ohjelmistotuotannon keinoin OA–ratkaisu tutkittavalle tapaukselle. Ratkaisun vaatimukset määritellään tarkastelemalla tapausta ja haastattelemalla kurssien henkilökuntaa. Kehitetyn ratkaisun avulla aineisto on saatavilla reaaliaikaisesti. Lisäksi ratkaisu mahdollistaa aineiston yhdistämisen Moodle ja A-plus oppimisympäristöistä yhteistä analyysiä varten. Työssä suunnitellaan uusia interaktiivisia tiedon visualisointeja käyttäjävaatimusten mukaisesti. Tutkimus OA:n käynnistämiseksi kurssitasolla tuotti kaksi yleistä tulosta. Ensiksi aineiston yhdistäminen eri lähteistä on tyypillinen haaste, joka vaatii suunnittelua. Toiseksi opettajien tavoitteita OA:ta aloittaessa ovat valvoa odotettua edistymistä, parantaa oppimateriaalin mitoitusta, tunnistaa ongelmakohtia oppimateriaalissa ja parantaa vuorovaikutusta opiskelijoiden kanssa

Improving serious games evaluation by applying learning analytics and data mining techniques

Tesis inédita de la Universidad Complutense de Madrid, Facultad de Informática, Departamento de Ingeniería del Software e Inteligencia Artificial, leída el 15/06/2017. Tesis formato europeo (compendio de artículos)Serious games are highly motivational resources effective to teach, raise awareness, or change the perceptions of players. To foster their application in education, teachers and institutions require clear and formal evidences to assess students' learning while they are playing the games. However, traditional assessment techniques rely on external questionnaires, typically carried out before and after playing, that fail to measure players' learning while it is happening. The multiple interactions carried out by players in the games can provide more precise information about how players play, and even be used to assess them. In this regard, game learning analytics techiques propose the collection and analysis of such interactions for multiple purposes, including assessment. The potentially large game learning analytics data collected can be further analyzed with data mining techniques to discover unexpected patterns and to provide measures to evaluate the effect of fames on their players and assess their learning...Los juegos serios son recursos altamente motivadores y efectivos para enseñar, concienciar, o cambiar las percepciones de sus jugadores. Para fomentar su aplicación en educación, los profesores y las instituciones necesitan pruebas claras y automáticas con las que evaluar el aprendizaje de sus estudiantes mientras utilizan los juegos. Tradicionalmente, la evaluación con juegos serios se basa en cuestionarios externos, realizados normalmente antes y después de jugar, que no miden el aprendizaje de los jugadores durante el proceso en sí. Las múltiples interacciones que realizan los jugadores al jugar pueden proporcionar una información más precisa sobre cómo juegan los jugadores e, incluso, utilizarse para evaluar su aprendizaje. En este sentido, las analíticas de aprendizaje para juegos proponen técnicas para la recogida y el análisis de dichas interacciones con múltiples fines, incluida la evaluación de los jugadores. Los datos (potencialmente numerosos) de las analíticas de aprendizaje para juegos pueden analizarse en mayor detalle con técnicas d minería de datos que permiten descubrir patrones ocultos a simple vista y proporcionar mejores medidas para estudiar el efecto de los juegos en los estudiantes y evaluar su aprendizaje...Fac. de InformáticaTRUEunpu

Analysis of user behavior with different interfaces in 360-degree videos and virtual reality

[eng] Virtual reality and its related technologies are being used for many kinds of content, like virtual environments or 360-degree videos. Omnidirectional, interactive, multimedia is consumed with a variety of devices, such as computers, mobile devices, or specialized virtual reality gear. Studies on user behavior with computer interfaces are an important part of the research in human-computer interaction, used in, e.g., studies on usability, user experience or the improvement of streaming techniques. User behavior in these environments has drawn the attention of the field but little attention has been paid to compare the behavior between different devices to reproduce virtual environments or 360-degree videos. We introduce an interactive system that we used to create and reproduce virtual reality environments and experiences based on 360-degree videos, which is able to automatically collect the users’ behavior, so we can analyze it. We studied the behavior collected in the reproduction of a virtual reality environment with this system and we found significant differences in the behavior between users of an interface based on the Oculus Rift and another based on a mobile VR headset similar to the Google Cardboard: different time between interactions, likely due to the need to perform a gesture in the first interface; differences in spatial exploration, as users of the first interface chose a particular area of the environment to stay; and differences in the orientation of their heads, as Oculus users tended to look towards physical objects in the experiment setup and mobile users seemed to be influenced by the initial values of orientation of their browsers. A second study was performed with data collected with this system, which was used to play a hypervideo production made of 360-degree videos, where we compared the users’ behavior with four interfaces (two based on immersive devices and the other two based on non-immersive devices) and with two categories of videos: we found significant differences in the spatiotemporal exploration, the dispersion of the orientation of the users, in the movement of these orientations and in the clustering of their trajectories, especially between different video types but also between devices, as we found that in some cases, behavior with immersive devices was similar due to similar constraints in the interface, which are not present in non-immersive devices, such as a computer mouse or the touchscreen of a smartphone. Finally, we report a model based on a recurrent neural network that is able to classify these reproductions with 360-degree videos into their corresponding video type and interface with an accuracy of more than 90% with only four seconds worth of orientation data; another deep learning model was implemented to predict orientations up to two seconds in the future from the last seconds of orientation, whose results were improved by up to 19% by a comparable model that leverages the video type and the device used to play it.[cat] La realitat virtual i les tecnologies que hi estan relacionades es fan servir per a molts tipus de continguts, com entorns virtuals o vídeos en 360 graus. Continguts multimèdia omnidireccional i interactiva són consumits amb diversos dispositius, com ordinadors, dispositius mòbils o aparells especialitzats de realitat virtual. Els estudis del comportament dels usuaris amb interfícies d’ordinador són una part important de la recerca en la interacció persona-ordinador fets servir en, per exemple, estudis de usabilitat, d’experiència d’usuari o de la millora de tècniques de transmissió de vídeo. El comportament dels usuaris en aquests entorns ha atret l’atenció dels investigadors, però s’ha parat poca atenció a comparar el comportament dels usuaris entre diferents dispositius per reproduir entorns virtuals o vídeos en 360 graus. Nosaltres introduïm un sistema interactiu que hem fet servir per crear i reproduir entorns de realitat virtual i experiències basades en vídeos en 360 graus, que és capaç de recollir automàticament el comportament dels usuaris, de manera que el puguem analitzar. Hem estudiat el comportament recollit en la reproducció d’un entorn de realitat virtual amb aquest sistema i hem trobat diferències significatives en l’execució entre usuaris d’una interfície basada en Oculus Rift i d’una altra basada en un visor de RV mòbil semblant a la Google Cardboard: diferent temps entre interaccions, probablement causat per la necessitat de fer un gest amb la primera interfície; diferències en l’exploració espacial, perquè els usuaris de la primera interfície van triar romandre en una àrea de l’entorn; i diferències en l’orientació dels seus caps, ja que els usuaris d’Oculus tendiren a mirar cap a objectes físics de la instal·lació de l’experiment i els usuaris dels visors mòbils semblen influïts pels valors d’orientació inicials dels seus navegadors. Un segon estudi va ser executat amb les dades recollides amb aquest sistema, que va ser fet servir per reproduir un hipervídeo fet de vídeos en 360 graus, en què hem comparat el comportament dels usuaris entre quatre interfícies (dues basades en dispositius immersius i dues basades en dispositius no immersius) i dues categories de vídeos: hem trobat diferències significatives en l’exploració de l’espaitemps del vídeo, en la dispersió de l’orientació dels usuaris, en el moviment d’aquestes orientacions i en l’agrupació de les seves trajectòries, especialment entre diferents tipus de vídeo però també entre dispositius, ja que hem trobat que, en alguns casos, el comportament amb dispositius immersius és similar a causa de límits semblants en la interfície, que no són presents en dispositius no immersius, com amb un ratolí d’ordinador o la pantalla tàctil d’un mòbil. Finalment, hem reportat un model basat en una xarxa neuronal recurrent, que és capaç de classificar aquestes reproduccions de vídeos en 360 graus en els seus corresponents tipus de vídeo i interfície que s’ha fet servir amb una precisió de més del 90% amb només quatre segons de trajectòria d’orientacions; un altre model d’aprenentatge profund ha estat implementat per predir orientacions fins a dos segons en el futur a partir dels darrers segons d’orientació, amb uns resultats que han estat millorats fins a un 19% per un model comparable que aprofita el tipus de vídeo i el dispositiu que s’ha fet servir per reproduir-lo.[spa] La realidad virtual y las tecnologías que están relacionadas con ella se usan para muchos tipos de contenidos, como entornos virtuales o vídeos en 360 grados. Contenidos multimedia omnidireccionales e interactivos son consumidos con diversos dispositivos, como ordenadores, dispositivos móviles o aparatos especializados de realidad virtual. Los estudios del comportamiento de los usuarios con interfaces de ordenador son una parte importante de la investigación en la interacción persona-ordenador usados en, por ejemplo, estudios de usabilidad, de experiencia de usuario o de la mejora de técnicas de transmisión de vídeo. El comportamiento de los usuarios en estos entornos ha atraído la atención de los investigadores, pero se ha dedicado poca atención en comparar el comportamiento de los usuarios entre diferentes dispositivos para reproducir entornos virtuales o vídeos en 360 grados. Nosotros introducimos un sistema interactivo que hemos usado para crear y reproducir entornos de realidad virtual y experiencias basadas en vídeos de 360 grados, que es capaz de recoger automáticamente el comportamiento de los usuarios, de manera que lo podamos analizar. Hemos estudiado el comportamiento recogido en la reproducción de un entorno de realidad virtual con este sistema y hemos encontrado diferencias significativas en la ejecución entre usuarios de una interficie basada en Oculus Rift y otra basada en un visor de RV móvil parecido a la Google Cardboard: diferente tiempo entre interacciones, probablemente causado por la necesidad de hacer un gesto con la primera interfaz; diferencias en la exploración espacial, porque los usuarios de la primera interfaz permanecieron en un área del entorno; y diferencias en la orientación de sus cabezas, ya que los usuarios de Oculus tendieron a mirar hacia objetos físicos en la instalación del experimento y los usuarios de los visores móviles parecieron influidos por los valores iniciales de orientación de sus navegadores. Un segundo estudio fue ejecutado con los datos recogidos con este sistema, que fue usado para reproducir un hipervídeo compuesto de vídeos en 360 grados, en el que hemos comparado el comportamiento de los usuarios entre cuatro interfaces (dos basadas en dispositivos inmersivos y dos basadas en dispositivos no inmersivos) y dos categorías de vídeos: hemos encontrado diferencias significativas en la exploración espaciotemporal del vídeo, en la dispersión de la orientación de los usuarios, en el movimiento de estas orientaciones y en la agrupación de sus trayectorias, especialmente entre diferentes tipos de vídeo pero también entre dispositivos, ya que hemos encontrado que, en algunos casos, el comportamiento con dispositivos inmersivos es similar a causa de límites parecidos en la interfaz, que no están presentes en dispositivos no inmersivos, como con un ratón de ordenador o la pantalla táctil de un móvil. Finalmente, hemos reportado un modelo basado en una red neuronal recurrente, que es capaz de clasificar estas reproducciones de vídeos en 360 grados en sus correspondientes tipos de vídeo y la interfaz que se ha usado con una precisión de más del 90% con sólo cuatro segundos de trayectoria de orientación; otro modelo de aprendizaje profundo ha sido implementad para predecir orientaciones hasta dos segundos en el futuro a partir de los últimos segundos de orientación, con unos resultados que han sido mejorados hasta un 19% por un modelo comparable que aprovecha el tipo de vídeo y el dispositivo que se ha usado para reproducirlo

Visual analysis of discrimination in machine learning

The growing use of automated decision-making in critical applications, such as crime prediction and college admission, has raised questions about fairness in machine learning. How can we decide whether different treatments are reasonable or discriminatory? In this paper, we investigate discrimination in machine learning from a visual analytics perspective and propose an interactive visualization tool, DiscriLens, to support a more comprehensive analysis. To reveal detailed information on algorithmic discrimination, DiscriLens identifies a collection of potentially discriminatory itemsets based on causal modeling and classification rules mining. By combining an extended Euler diagram with a matrix-based visualization, we develop a novel set visualization to facilitate the exploration and interpretation of discriminatory itemsets. A user study shows that users can interpret the visually encoded information in DiscriLens quickly and accurately. Use cases demonstrate that DiscriLens provides informative guidance in understanding and reducing algorithmic discrimination

An Instructional Designer Competency Framework for Complex Learning Designs

Learning design competency frameworks published by professional organizations, exist for typical instructional design efforts. However, a review of literature revealed a lack of frameworks available for the creation of complex learning designs (CLDs). The goal of this research was to develop a competency framework for the creation of CLDs. Quantitative and qualitative methods were employed in the four phases of the design and development research approach In phase one, a survey based on the Educational Technology Multimedia Competency Survey (ETMCS) was sent to instructional designers who self-reported as having experience creating CLDs. The purpose of phase one was to identify competencies that instructional designers felt were most important to the creation of complex, technology-mediated learning designs. The preliminary CLD framework was constructed during phase two, based on analysis of the ETMCS survey results. Measures of central tendency were used to identify competencies considered essential and desirable. Additionally, competencies were categorized into seven domains In phase three, semi-structured interviews were conducted with a subset of survey participants. The purpose was to gain deeper insight into the participant’s perception of the design complexities involved with each of the competencies included in the preliminary framework. In phase four, the preliminary framework was internally validated using an expert panel employing the Delphi method to build consensus. Three rounds were required to achieve consensus on all competencies within the framework. This consensus resulted in 79 competencies including 30 essential and 49 desirable competencies from the set identified as the preliminary framework during phase two. Several conclusions emerged from the creation of this framework. Though technology is often a trigger for many types of CLDs, specific technologies are certainly desirable, but not essential. The research also revealed that communication and collaboration competencies are almost universally essential due to the complexity of the designs which typically necessitates the formation of multi-discipline teams. Without these competencies, the team’s cross-profession effectiveness is often hindered due to differences in terminology, processes, and team member geographic location

Survol des situations de travail, des processus de production, du contrôle de la qualité et des bonnes pratiques en conception et réalisation d'outils pédagogiques en ligne /

Comprend des références bibliographiques

Exploring the role of experience API in supporting new trends in Educational Technology: A literature review

Despite its growth in the area of educational technology, Experience API (xAPI) continues to be under-used as a solution across the different platforms in institutions and organizations. There is a lack of any detailed summary in the literature about the potential and the limitation of using xAPI in conjunction with learning platforms and technologies. This thesis examines the role of xAPI in promoting, shaping and supporting learning in organizational contexts. This discussion is developed by using cases reported in the literature and new cases from contemporary educational technologies. The thesis illustrates the role the standard plays within current major trends in digital learning and within the context of a broader ecosystem of learning platforms and technologies. It provides a useful and thorough account of xAPI and its potential to an audience of individuals responsible for implementing xAPI within organizations. xAPI provides to some extent a promise of improved impact to Performance Evaluation and Evaluating training Effectiveness. However, xAPI lacks concrete cases and examples to support its utilization in the fields of Learning Analytics, Performance Management, Predictive Learning and Workforce Planning. Keywords: Experience API (xAPI), Learning Management Systems, Learning Record Store, Learning Analytics, Microlearning, Evaluation Effectiveness, Predictive Learning, Adaptive Learning, Workforce Planning