Heuristic Evaluation for Serious Immersive Games and M-instruction

© Springer International Publishing Switzerland 2016. Two fast growing areas for technology-enhanced learning are serious games and mobile instruction (M-instruction or M-Learning). Serious games are ones that are meant to be more than just entertainment. They have a serious use to educate or promote other types of activity. Immersive Games frequently involve many players interacting in a shared rich and complex-perhaps web-based-mixed reality world, where their circumstances will be multi and varied. Their reality may be augmented and often self-composed, as in a user-defined avatar in a virtual world. M-instruction and M-Learning is learning on the move; much of modern computer use is via smart devices, pads, and laptops. People use these devices all over the place and thus it is a natural extension to want to use these devices where they are to learn. This presents a problem if we wish to evaluate the effectiveness of the pedagogic media they are using. We have no way of knowing their situation, circumstance, education background and motivation, or potentially of the customisation of the final software they are using. Getting to the end user itself may also be problematic; these are learning environments that people will dip into at opportune moments. If access to the end user is hard because of location and user self-personalisation, then one solution is to look at the software before it goes out. Heuristic Evaluation allows us to get User Interface (UI) and User Experience (UX) experts to reflect on the software before it is deployed. The effective use of heuristic evaluation with pedagogical software [1] is extended here, with existing Heuristics Evaluation Methods that make the technique applicable to Serious Immersive Games and mobile instruction (M-instruction). We also consider how existing Heuristic Methods may be adopted. The result represents a new way of making this methodology applicable to this new developing area of learning technology

'I play, therefore I learn?' Measuring the Evolution of Perceived Learning and Game Experience in the Design Flow of a Serious Game

This article explores how the serious game Poverty Is Not a Game (PING) is experienced by high school students in its subsequent design stages. We first focus on the multifaceted construct of game experience and how it is related to serious games. To measure game experience we use the Game Experience Questionnaire and add a perceived learning scale to account for the specificity of serious games in a classroom. Next, the data obtained from testing PING in 22 classrooms are analyzed. Results suggest that the evolution in the different design stages of the game is not just an issue of game experience, but also of usability. Furthermore, little evidence is found indicating that the learning experience changed positively during the different test phases. However, findings show a strong effect of the game experience on perceived learning while the game experience also varies significantly between different classrooms

Understanding the fidelity effect when evaluating games with children

There have been a number of studies that have compared evaluation results from prototypes of different fidelities but very few of these are with children. This paper reports a comparative study of three prototypes ranging from low fidelity to high fidelity within the context of mobile games, using a between subject design with 37 participants aged 7 to 9. The children played a matching game on either an iPad, a paper prototype using screen shots of the actual game or a sketched version. Observational data was captured to establish the usability problems, and two tools from the Fun Toolkit were used to measure user experience. The results showed that there was little difference for user experience between the three prototypes and very few usability problems were unique to a specific prototype. The contribution of this paper is that children using low-fidelity prototypes can effectively evaluate games of this genre and style

The Limited Effect of Graphic Elements in Video and Augmented Reality on Children’s Listening Comprehension

There is currently significant interest in the use of instructional strategies in learning environments thanks to the emergence of new multimedia systems that combine text, audio, graphics and video, such as augmented reality (AR). In this light, this study compares the effectiveness of AR and video for listening comprehension tasks. The sample consisted of thirty-two elementary school students with different reading comprehension. Firstly, the experience, instructions and objectives were introduced to all the students. Next, they were divided into two groups to perform activities—one group performed an activity involving watching an Educational Video Story of the Laika dog and her Space Journey available by mobile devices app Blue Planet Tales, while the other performed an activity involving the use of AR, whose contents of the same history were visualized by means of the app Augment Sales. Once the activities were completed participants answered a comprehension test. Results (p = 0.180) indicate there are no meaningful differences between the lesson format and test performance. But there are differences between the participants of the AR group according to their reading comprehension level. With respect to the time taken to perform the comprehension test, there is no significant difference between the two groups but there is a difference between participants with a high and low level of comprehension. To conclude SUS (System Usability Scale) questionnaire was used to establish the measure usability for the AR app on a smartphone. An average score of 77.5 out of 100 was obtained in this questionnaire, which indicates that the app has fairly good user-centered design

Bringing tabletop technologies to kindergarten children

Taking computer technology away from the desktop and into a more physical, manipulative space, is known that provide many benefits and is generally considered to result in a system that is easier to learn and more natural to use. This paper describes a design solution that allows kindergarten children to take the benefits of the new pedagogical possibilities that tangible interaction and tabletop technologies offer for manipulative learning. After analysis of children's cognitive and psychomotor skills, we have designed and tuned a prototype game that is suitable for children aged 3 to 4 years old. Our prototype uniquely combines low cost tangible interaction and tabletop technology with tutored learning. The design has been based on the observation of children using the technology, letting them freely play with the application during three play sessions. These observational sessions informed the design decisions for the game whilst also confirming the children's enjoyment of the prototype

Evaluating 2D and 3D serious games: The significance of student-player characteristics

Τα σοβαρά παιχνίδια (ψηφιακά εκπαιδευτικά παιχνίδια μαθησιακού σκοπού)  κερδίζουν ολοένα και περισσότερο το ενδιαφέρον πολλών μελετητών της μαθησιακής διαδικασίας όλων των εκπαιδευτικών επιπέδων. Σημαντικές πτυχές της εφαρμογής τους στην εκπαίδευση σχετίζονται με τα χαρακτηριστικά τους, τη συνάφειά τους με την παιδαγωγική και τις απόψεις των μαθητών. Αυτή η ποσοτική έρευνα επικεντρώθηκε στα χαρακτηριστικά των φοιτητών παιδαγωγικών τμημάτων (φύλο, επιστημονικό υπόβαθρο, προηγούμενη εμπειρία με παιχνίδια και δεξιότητες ΤΠΕ) και στους παράγοντες που επηρεάζουν και διαμορφώνουν την μαθησιακή εμπειρία τους όταν παίζουν σοβαρά παιχνίδια δύο (2D) και τριών (3D) διαστάσεων, όπως είναι η εμβύθιση, η απόλαυση, η υποκειμενική αντίληψη των μαθητών σχετικά με τη χρησιμότητά τους για τη βελτίωση της γνώσης, την επάρκεια της αφήγησης, τον ρεαλισμό, την ανατροφοδότηση, την οπτικοακουστική επάρκεια, τη συνάφειά τους με τα προσωπικά ενδιαφέροντά τους, τη σαφήνεια του στόχου τους, την ευκολία χρήσης τους, την καταλληλότητα του μαθησιακού υλικού και τα κίνητρα. Συνολικά 542 φοιτητές συμμετείχαν στη μελέτη. Σειρά μη παραμετρικών αναλύσεων (Mann-Whitney U tests και Kruskal-Wallis H tests) αποκάλυψε ότι οι φοιτητές προτίμησαν το 2D παιχνίδι. Το φύλο, το επιστημονικό υπόβαθρο και οι δεξιότητες στις ΤΠΕ δεν είχαν αντίκτυπο στις απόψεις τους. Από την άλλη πλευρά, οι φοιτητές με υψηλή ικανότητα στη χρήση των παιχνιδιών φαίνεται να έχουν καλύτερη μαθησιακή εμπειρία, καθώς οι βαθμολογίες τους στους περισσότερους παράγοντες ήταν υψηλότερες σε σύγκριση με τους φοιτητές με χαμηλότερη ικανότητα. Αυτά τα αποτελέσματα μας οδήγησαν να προσανατολιστούμε στην εξέταση άλλων πτυχών/δεξιοτήτων των φοιτητών πέρα από τα συνήθη δημογραφικά στοιχεία, όπως είναι η αυτορρύθμιση, η χωρική ικανότητα και η νοητική περιστροφή και να ερευνήσουμε τη δυνατότητα των σοβαρών παιχνιδιών να βελτιώσουν αυτές τις δεξιότητες.Serious games are gaining ground in all educational levels. Important aspects of their implementation in education are related to their features, their relevance to pedagogy and the learners' views. This quantitative research focused on characteristics of pedagogical departments’ students (gender, scientific background, prior gaming and ICT skills) and factors are considered to affect and shape their learning experience when playing 2D and 3D serious games (immersion, enjoyment, perceived usefulness-knowledge improvement, perceived narratives' adequacy, perceived realism, perceived feedback's adequacy, perceived audiovisual adequacy, perceived relevance to personal interests, perceived goal's clarity, perceived ease of use, adequacy of the learning material and motivation). A total of 542 university students participated in the study. The data analysis revealed that they preferred the 2D game. Their gender, scientific background and ICT skills did not have an impact on their views. On the other hand, students with high game-playing competence seem to have a better learning experience, as their scores in most of the factors were higher compared to the ones with lower game-playing competence. Those results led us to consider other aspects of learners beyond ordinary demographics such as self-regulation, spatial cognition and mental rotation and to examine the evidence that serious games can improve these skills

Evaluating 2D and 3D serious games: The significance of student-player characteristics

Τα σοβαρά παιχνίδια (ψηφιακά εκπαιδευτικά παιχνίδια μαθησιακού σκοπού)  κερδίζουν ολοένα και περισσότερο το ενδιαφέρον πολλών μελετητών της μαθησιακής διαδικασίας όλων των εκπαιδευτικών επιπέδων. Σημαντικές πτυχές της εφαρμογής τους στην εκπαίδευση σχετίζονται με τα χαρακτηριστικά τους, τη συνάφειά τους με την παιδαγωγική και τις απόψεις των μαθητών. Αυτή η ποσοτική έρευνα επικεντρώθηκε στα χαρακτηριστικά των φοιτητών παιδαγωγικών τμημάτων (φύλο, επιστημονικό υπόβαθρο, προηγούμενη εμπειρία με παιχνίδια και δεξιότητες ΤΠΕ) και στους παράγοντες που επηρεάζουν και διαμορφώνουν την μαθησιακή εμπειρία τους όταν παίζουν σοβαρά παιχνίδια δύο (2D) και τριών (3D) διαστάσεων, όπως είναι η εμβύθιση, η απόλαυση, η υποκειμενική αντίληψη των μαθητών σχετικά με τη χρησιμότητά τους για τη βελτίωση της γνώσης, την επάρκεια της αφήγησης, τον ρεαλισμό, την ανατροφοδότηση, την οπτικοακουστική επάρκεια, τη συνάφειά τους με τα προσωπικά ενδιαφέροντά τους, τη σαφήνεια του στόχου τους, την ευκολία χρήσης τους, την καταλληλότητα του μαθησιακού υλικού και τα κίνητρα. Συνολικά 542 φοιτητές συμμετείχαν στη μελέτη. Σειρά μη παραμετρικών αναλύσεων (Mann-Whitney U tests και Kruskal-Wallis H tests) αποκάλυψε ότι οι φοιτητές προτίμησαν το 2D παιχνίδι. Το φύλο, το επιστημονικό υπόβαθρο και οι δεξιότητες στις ΤΠΕ δεν είχαν αντίκτυπο στις απόψεις τους. Από την άλλη πλευρά, οι φοιτητές με υψηλή ικανότητα στη χρήση των παιχνιδιών φαίνεται να έχουν καλύτερη μαθησιακή εμπειρία, καθώς οι βαθμολογίες τους στους περισσότερους παράγοντες ήταν υψηλότερες σε σύγκριση με τους φοιτητές με χαμηλότερη ικανότητα. Αυτά τα αποτελέσματα μας οδήγησαν να προσανατολιστούμε στην εξέταση άλλων πτυχών/δεξιοτήτων των φοιτητών πέρα από τα συνήθη δημογραφικά στοιχεία, όπως είναι η αυτορρύθμιση, η χωρική ικανότητα και η νοητική περιστροφή και να ερευνήσουμε τη δυνατότητα των σοβαρών παιχνιδιών να βελτιώσουν αυτές τις δεξιότητες.Serious games are gaining ground in all educational levels. Important aspects of their implementation in education are related to their features, their relevance to pedagogy and the learners' views. This quantitative research focused on characteristics of pedagogical departments’ students (gender, scientific background, prior gaming and ICT skills) and factors are considered to affect and shape their learning experience when playing 2D and 3D serious games (immersion, enjoyment, perceived usefulness-knowledge improvement, perceived narratives' adequacy, perceived realism, perceived feedback's adequacy, perceived audiovisual adequacy, perceived relevance to personal interests, perceived goal's clarity, perceived ease of use, adequacy of the learning material and motivation). A total of 542 university students participated in the study. The data analysis revealed that they preferred the 2D game. Their gender, scientific background and ICT skills did not have an impact on their views. On the other hand, students with high game-playing competence seem to have a better learning experience, as their scores in most of the factors were higher compared to the ones with lower game-playing competence. Those results led us to consider other aspects of learners beyond ordinary demographics such as self-regulation, spatial cognition and mental rotation and to examine the evidence that serious games can improve these skills