Heuristic usability evaluation on games: a modular approach

Heuristic evaluation is the preferred method to assess usability in games when experts conduct this evaluation. Many heuristics guidelines have been proposed attending to specificities of games but they only focus on specific subsets of games or platforms. In fact, to date the most used guideline to evaluate games usability is still Nielsen’s proposal, which is focused on generic software. As a result, most evaluations do not cover important aspects in games such as mobility, multiplayer interactions, enjoyability and playability, etc. To promote the usage of new heuristics adapted to different game and platform aspects we propose a modular approach based on the classification of existing game heuristics using metadata and a tool, MUSE (Meta-heUristics uSability Evaluation tool) for games, which allows a rebuild of heuristic guidelines based on metadata selection in order to obtain a customized list for every real evaluation case. The usage of these new rebuilt heuristic guidelines allows an explicit attendance to a wide range of usability aspects in games and a better detection of usability issues. We preliminarily evaluate MUSE with an analysis of two different games, using both the Nielsen’s heuristics and the customized heuristic lists generated by our tool.Unión Europea PI055-15/E0

Methodologies for evaluating the playability of mobile games:systematic literature review

Tiivistelmä. The gaming industry has been growing rapidly during the past years due to the interest of the new generations in mobile gaming. To deliver a great experience for the gamers, it is required for the gaming companies to produce games that are challenging but at the same time easy to play. To achieve this, it is required to understand the factors that affect the gaming experience. Playability is a term that is used to understand the usability of a game and its experience. The purpose of this thesis was to understand what is known related to the playability of mobile games and to identify the methodologies that are used by the community to evaluate this phenomenon. To find the answers to these questions, it was performed a systematic literature review (SLR) using the databases Scopus, IEEE Xplore, and Web of Science. After conducting the SLR, 1,390 studies related to the playability of mobile games were found from which 27 were identified as primary studies of this research. From the data collected from the primary studies, there were identified 12 different methodologies that are used for evaluating the playability of mobile games. The methodologies that are most suitable to assess the playability of mobile games are heuristic evaluation and playtesting. Other methodologies can be used for evaluating the playability of mobile games, but they must include a set of heuristics that allows evaluating the playability. The limitations of the research were mentioned, and it was proposed topics for future research of this field. The contribution of this thesis is the summarizing of the current methodologies that are used to understand and evaluate the playability of mobile games. The results of this thesis are valuable for game developers, game designers, and game usability practitioners

Heuristic Evaluation for Gameful Design

© Lennart Nacke, 2016. This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The definitive version was published in CHI PLAY Companion '16 Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, https://doi.org/10.1145/2968120.2987729Despite the emergence of many gameful design methods in the literature, there is a lack of evaluation methods specific to gameful design. To address this gap, we present a new set of guidelines for heuristic evaluation of gameful design in interactive systems. First, we review several gameful design methods to identify the dimensions of motivational affordances most often employed. Then, we present a set of 28 gamification heuristics aimed at enabling experts to rapidly evaluate a gameful system. The resulting heuristics are a new method to evaluate user experience in gameful interactive systemsNatural Sciences and Engineering Research Council of Canada Social Sciences and Humanities Research Council of CanadaPeer-reviewe

Heuristics for evaluating the usability of CAA applications

Evaluation of usability is well researched in the area of HCI. One widely used method is a heuristic evaluation which relies on a small number of evaluators inspecting an interface to see to what extent it complies with a set of heuristics. Once a problem is identified it is categorised to a heuristic and a severity rating is attached. Severity ratings indicate the potential impact of the problem. Using a corpus of usability problems within CAA this paper reports on the development of domain specific heuristics and severity ratings for evaluating the usability of CAA applications. The heuristics are presented and the paper concludes with practical guidance on the application of the method in CAA

PLAYABILITY HEURISTIC EVALUATION SYSTEM FOR MOBILE GAMES

Playability is an element that measures the ease of use by which a game can be played. To evaluate playability of a game, two methods are widely being used; Playtesting and Heuristic Evaluation. Playability Heuristics are required to perform heuristic evaluation on games

Establishing usability heuristics for heuristics evaluation in a specific domain: is there a consensus?

Heuristics evaluation is frequently employed to evaluate usability. While general heuristics are suitable to evaluate most user interfaces, there is still a need to establish heuristics for specific domains to ensure that their specific usability issues are identified. This paper presents a comprehensive review of 70 studies related to usability heuristics for specific domains. The aim of this paper is to review the processes that were applied to establish heuristics in specific domains and identify gaps in order to provide recommendations for future research and area of improvements. The most urgent issue found is the deficiency of validation effort following heuristics proposition and the lack of robustness and rigour of validation method adopted. Whether domain specific heuristics perform better or worse than general ones is inconclusive due to lack of validation quality and clarity on how to assess the effectiveness of heuristics for specific domains. The lack of validation quality also affects effort in improving existing heuristics for specific domain as their weaknesses are not addressed

Heuristics for evaluating video games: A two-tier set incorporating universal and genre-specific elements

The continued significance of video games in contemporary society has led to the production of many heuristic sets by which the playability of games can be evaluated. These sets encompass a wide range of aims and theoretical perspectives, furthermore, they have been developed via a number of distinct methods. This thesis is the result of a two-stage investigation, the first of which was to identify and extract any heuristics that could be considered as having universal relevance, irrespective of the method or theoretical position by which they were developed. The second stage of the investigation complemented this universal list with genre-specific elements, providing a model by which further information can be added in the future. Comparative analysis was used to extract common principles from a selected body of work, resulting in a list of 19 heuristics. In addition, open and axial coding was applied to data sourced from online game reviews, extracting information relating to the Real-Time Strategy genre. This information was then converted into 25 RTS-specific heuristics and integrated with the set developed in the first stage. The results of this study, therefore, clearly demonstrate the existence of a core set of universal heuristics, and the value of incorporating genre-specific information. The finalised set was presented in the form of a deck of cards, with each face representing information according to two different levels of abstraction, thereby increasing accessibility and facilitating use in different contexts. These cards have been made available in a print-ready format and are included as an appendix

Optimization of Educational Mobile Game Design 'Ayo Wisata ke Riau' based on User's Perspective

Educational Mobile Game ‘Ayo Wisata ke Riau’ is a mobile-based educational game application that was developed in 2017 on the Android platform. The quality requirements used in this game development have been adjusted for the available Educational Mobile Games. Even though it went well, there were many complaints from users during the direct implementation survey. This becomes material for re-evaluation, whether the game is ready to be disseminated to the public. Based on this occurence, a gap analysis was carried out to see the feasibility and the need for improvements to the game application that had been built. Based on the results of the gap analysis that has been carried out, it is indicated that there is a need for significant improvements in the areas of performance and user satisfaction. So that in this study, a design reengineering was carried out that was focused on improving the two things that were considered crucial, so that the results of the design and implementation that had been improved could provide better performance and user satisfaction scores than previous developments.Educational Mobile Game ‘Ayo Wisata ke Riau’ is a mobile-based educational game application that was developed in 2017 on the Android platform. The quality requirements used in this game development have been adjusted for the available Educational Mobile Games. Even though it went well, there were many complaints from users during the direct implementation survey. This becomes material for re-evaluation, whether the game is ready to be disseminated to the public. Based on this occurence, a gap analysis was carried out to see the feasibility and the need for improvements to the game application that had been built. Based on the results of the gap analysis that has been carried out, it is indicated that there is a need for significant improvements in the areas of performance and user satisfaction. So that in this study, a design reengineering was carried out that was focused on improving the two things that were considered crucial, so that the results of the design and implementation that had been improved could provide better performance and user satisfaction scores than previous developments

The playthrough evaluation framework reliable usability evaluation for video games

This thesis presents the playthrough evaluation framework, a novel framework for the reliable usability evaluation of first-person shooter console video games. The framework includes playthrough evaluation, a structured usability evaluation method adapted from heuristic evaluation. Usability evaluation can help guide developers by pointing out design issues that cause users problems. However, usability evaluation methods suffer from the evaluator effect, where separate evaluations of the same data do not produce reliably consistent results. This can result in a number of undesirable consequences affecting issues such as: • Unreliable evaluation: Without reliable results, evaluation reports risk giving incorrect or misleading advice. • Weak methodological validation: Typically new methods (e.g., new heuristics) are validated against user tests. However, without a reliable means to describe observations, attempts to validate novel methods against user test data will also be affected by weak reliability. The playthrough evaluation framework addresses these points through a series of studies presenting the need for, and showing the development of the framework, including the following stages, 1. Explication of poor reliability in heuristic evaluation. 2. Development and validation of a reliable user test coding scheme. 3. Derivation of a novel usability evaluation method, playthrough evaluation. 4. Testing the method, quantifying results. Evaluations were conducted with 22 participants, on 3 first-person shooter action console video games, using two methodologies, heuristic evaluation and the novel playthrough evaluation developed in this thesis. Both methods proved effective, with playthrough evaluation providing more detailed analysis but requiring more time to conduct

Evaluating 2D and 3D serious games: The significance of student-player characteristics

Τα σοβαρά παιχνίδια (ψηφιακά εκπαιδευτικά παιχνίδια μαθησιακού σκοπού)  κερδίζουν ολοένα και περισσότερο το ενδιαφέρον πολλών μελετητών της μαθησιακής διαδικασίας όλων των εκπαιδευτικών επιπέδων. Σημαντικές πτυχές της εφαρμογής τους στην εκπαίδευση σχετίζονται με τα χαρακτηριστικά τους, τη συνάφειά τους με την παιδαγωγική και τις απόψεις των μαθητών. Αυτή η ποσοτική έρευνα επικεντρώθηκε στα χαρακτηριστικά των φοιτητών παιδαγωγικών τμημάτων (φύλο, επιστημονικό υπόβαθρο, προηγούμενη εμπειρία με παιχνίδια και δεξιότητες ΤΠΕ) και στους παράγοντες που επηρεάζουν και διαμορφώνουν την μαθησιακή εμπειρία τους όταν παίζουν σοβαρά παιχνίδια δύο (2D) και τριών (3D) διαστάσεων, όπως είναι η εμβύθιση, η απόλαυση, η υποκειμενική αντίληψη των μαθητών σχετικά με τη χρησιμότητά τους για τη βελτίωση της γνώσης, την επάρκεια της αφήγησης, τον ρεαλισμό, την ανατροφοδότηση, την οπτικοακουστική επάρκεια, τη συνάφειά τους με τα προσωπικά ενδιαφέροντά τους, τη σαφήνεια του στόχου τους, την ευκολία χρήσης τους, την καταλληλότητα του μαθησιακού υλικού και τα κίνητρα. Συνολικά 542 φοιτητές συμμετείχαν στη μελέτη. Σειρά μη παραμετρικών αναλύσεων (Mann-Whitney U tests και Kruskal-Wallis H tests) αποκάλυψε ότι οι φοιτητές προτίμησαν το 2D παιχνίδι. Το φύλο, το επιστημονικό υπόβαθρο και οι δεξιότητες στις ΤΠΕ δεν είχαν αντίκτυπο στις απόψεις τους. Από την άλλη πλευρά, οι φοιτητές με υψηλή ικανότητα στη χρήση των παιχνιδιών φαίνεται να έχουν καλύτερη μαθησιακή εμπειρία, καθώς οι βαθμολογίες τους στους περισσότερους παράγοντες ήταν υψηλότερες σε σύγκριση με τους φοιτητές με χαμηλότερη ικανότητα. Αυτά τα αποτελέσματα μας οδήγησαν να προσανατολιστούμε στην εξέταση άλλων πτυχών/δεξιοτήτων των φοιτητών πέρα από τα συνήθη δημογραφικά στοιχεία, όπως είναι η αυτορρύθμιση, η χωρική ικανότητα και η νοητική περιστροφή και να ερευνήσουμε τη δυνατότητα των σοβαρών παιχνιδιών να βελτιώσουν αυτές τις δεξιότητες.Serious games are gaining ground in all educational levels. Important aspects of their implementation in education are related to their features, their relevance to pedagogy and the learners' views. This quantitative research focused on characteristics of pedagogical departments’ students (gender, scientific background, prior gaming and ICT skills) and factors are considered to affect and shape their learning experience when playing 2D and 3D serious games (immersion, enjoyment, perceived usefulness-knowledge improvement, perceived narratives' adequacy, perceived realism, perceived feedback's adequacy, perceived audiovisual adequacy, perceived relevance to personal interests, perceived goal's clarity, perceived ease of use, adequacy of the learning material and motivation). A total of 542 university students participated in the study. The data analysis revealed that they preferred the 2D game. Their gender, scientific background and ICT skills did not have an impact on their views. On the other hand, students with high game-playing competence seem to have a better learning experience, as their scores in most of the factors were higher compared to the ones with lower game-playing competence. Those results led us to consider other aspects of learners beyond ordinary demographics such as self-regulation, spatial cognition and mental rotation and to examine the evidence that serious games can improve these skills