Academic methods for usability evaluation of serious games: a systematic review

In the last years, there has been an increasing interest in the design of video games as a tool for education, training, health promotion, socialization, etc. Usability, which is a key factor in any video game, becomes even more important in these so-called Bserious games^, where the users’ special characteristics should be considered, and the game efficacy depends on the users’ adherence and engagement. However, evaluation of the usability of this kind of games requires a redefinition of techniques, methods and even terminology. In this paper, we elicit six research questions and conduct a systematic review of the scientific literature, which resulted in the selection of 187 papers that contained the most relevant responses. The conclusions of this systematic review illustrate the general status of current academic usability evaluations of these games and the main trends in the selection of methodologies and how are they applied. This view may be a very valuable foundation for future researchMinisterio de Ciencia e Innovación PROCUR@-IPT-2011-1038-90000

The playthrough evaluation framework reliable usability evaluation for video games

This thesis presents the playthrough evaluation framework, a novel framework for the reliable usability evaluation of first-person shooter console video games. The framework includes playthrough evaluation, a structured usability evaluation method adapted from heuristic evaluation. Usability evaluation can help guide developers by pointing out design issues that cause users problems. However, usability evaluation methods suffer from the evaluator effect, where separate evaluations of the same data do not produce reliably consistent results. This can result in a number of undesirable consequences affecting issues such as: • Unreliable evaluation: Without reliable results, evaluation reports risk giving incorrect or misleading advice. • Weak methodological validation: Typically new methods (e.g., new heuristics) are validated against user tests. However, without a reliable means to describe observations, attempts to validate novel methods against user test data will also be affected by weak reliability. The playthrough evaluation framework addresses these points through a series of studies presenting the need for, and showing the development of the framework, including the following stages, 1. Explication of poor reliability in heuristic evaluation. 2. Development and validation of a reliable user test coding scheme. 3. Derivation of a novel usability evaluation method, playthrough evaluation. 4. Testing the method, quantifying results. Evaluations were conducted with 22 participants, on 3 first-person shooter action console video games, using two methodologies, heuristic evaluation and the novel playthrough evaluation developed in this thesis. Both methods proved effective, with playthrough evaluation providing more detailed analysis but requiring more time to conduct

Piloting Multimodal Learning Analytics using Mobile Mixed Reality in Health Education

© 2019 IEEE. Mobile mixed reality has been shown to increase higher achievement and lower cognitive load within spatial disciplines. However, traditional methods of assessment restrict examiners ability to holistically assess spatial understanding. Multimodal learning analytics seeks to investigate how combinations of data types such as spatial data and traditional assessment can be combined to better understand both the learner and learning environment. This paper explores the pedagogical possibilities of a smartphone enabled mixed reality multimodal learning analytics case study for health education, focused on learning the anatomy of the heart. The context for this study is the first loop of a design based research study exploring the acquisition and retention of knowledge by piloting the proposed system with practicing health experts. Outcomes from the pilot study showed engagement and enthusiasm of the method among the experts, but also demonstrated problems to overcome in the pedagogical method before deployment with learners

Structured computer-based training in the interpretation of neuroradiological images

Computer-based systems may be able to address a recognised need throughout the medical profession for a more structured approach to training. We describe a combined training system for neuroradiology, the MR Tutor that differs from previous approaches to computer-assisted training in radiology in that it provides case-based tuition whereby the system and user communicate in terms of a well-founded Image Description Language. The system implements a novel method of visualisation and interaction with a library of fully described cases utilising statistical models of similarity, typicality and disease categorisation of cases. We describe the rationale, knowledge representation and design of the system, and provide a formative evaluation of its usability and effectiveness

The design-by-adaptation approach to universal access: learning from videogame technology

This paper proposes an alternative approach to the design of universally accessible interfaces to that provided by formal design frameworks applied ab initio to the development of new software. This approach, design-byadaptation, involves the transfer of interface technology and/or design principles from one application domain to another, in situations where the recipient domain is similar to the host domain in terms of modelled systems, tasks and users. Using the example of interaction in 3D virtual environments, the paper explores how principles underlying the design of videogame interfaces may be applied to a broad family of visualization and analysis software which handles geographical data (virtual geographic environments, or VGEs). One of the motivations behind the current study is that VGE technology lags some way behind videogame technology in the modelling of 3D environments, and has a less-developed track record in providing the variety of interaction methods needed to undertake varied tasks in 3D virtual worlds by users with varied levels of experience. The current analysis extracted a set of interaction principles from videogames which were used to devise a set of 3D task interfaces that have been implemented in a prototype VGE for formal evaluation

Information Technologies for Cognitive Decline

Information technology (IT) is used to establish a diagnosis and provide treatment for people with cognitive decline. The condition affects many before it becomes clear that more permanent changes, like dementia, could be noticed. Those who search for information are exposed to lots of information and different technologies which they need to make sense of and eventually use to help themselves. In this research literature and information available on the Internet were systematically analyzed to present methods used for diagnosis and treatment. Methods used for diagnosis are self-testing, sensors, Virtual Reality (VR), and brain imaging. Methods used for treatment are games, websites with information and media, Virtual Reality (VR), sensors, and robots. The resulting concept of knowledge was the basis of an artifact whose main goal was to present the facts to the broad public. This implied that a user-friendly artifact was developed through three iterations using the Design Science framework. A total of nine users and IT usability experts have evaluated the artifact returning the SUS score of 85,83 for users and 87,5 for IT usability experts. Nielsen´s heuristics were assessed by IT usability experts only, returning an average score of 4,28. The general response was positive regarding both the content and the attempt to present methods used in cognitive decline. It reminds to be seen how to bring this knowledge to those who are most affected by the decline.Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Supporting the Mobile In-situ Authoring of Locative Media in Rural Places: Design and Expert Evaluation of the SMAT app

Providing users with carefully authored Locative media experiences (which can be consumed via their GPS equipped smartphones or tablets) has significant potential for fostering a strong engagement with their current surroundings. However, the availability of mobile tools to support the authoring of locative media experiences in-situ, and by non-technical users, remains scarce. In this article we present the design and field-trial expert evaluation of a mobile app developed under the SHARC project (Investigating Technology Support for the Shared Curation of Local History in a Rural Community). The app is named SMAT (SHARC Mobile Authoring Tool) and supports the authoring of Locative Media experiences with a focus on the creation of POIs (Points of Interest) and associated geo-fences which trigger the pushed delivery of media items such as photos, audio clips, etc. One important requirement of SMAT is the ability to support authoring in places where connectivity is intermittent or unavailable, e.g. many rural areas