Academic methods for usability evaluation of serious games: a systematic review

In the last years, there has been an increasing interest in the design of video games as a tool for education, training, health promotion, socialization, etc. Usability, which is a key factor in any video game, becomes even more important in these so-called Bserious games^, where the users’ special characteristics should be considered, and the game efficacy depends on the users’ adherence and engagement. However, evaluation of the usability of this kind of games requires a redefinition of techniques, methods and even terminology. In this paper, we elicit six research questions and conduct a systematic review of the scientific literature, which resulted in the selection of 187 papers that contained the most relevant responses. The conclusions of this systematic review illustrate the general status of current academic usability evaluations of these games and the main trends in the selection of methodologies and how are they applied. This view may be a very valuable foundation for future researchMinisterio de Ciencia e Innovación PROCUR@-IPT-2011-1038-90000

DeReFrame: a design-research framework to study game mechanics and game aesthetics in an engineering design process

The main aim of this research is to study gaming techniques and elements that may potentially be beneficial to the future development of CAD systems for engineering design, in particular to maintain cognitive engagement. A design-research framework, called DeReFrame, was employed to construct an experimental game-based CAD framework exploring this. This research is based on reviews from the literature and experimental studies and include quantitative and qualitative data analysis methods measuring engineers’ performance and emotional responses. The thesis presents the construction process of the framework (DeReframe) to study a set of game mechanics and game aesthetics in an engineering design process and compare this with the traditional CAD. The framework was used to design and implement a game-based CAD system, called ICAD which was embedded with the following game mechanics of Directional Goals, Progression, Performance-Feedback and Rewards-Achievement. The DeReFrame and ICAD evolved through the experimental studies. In each case, selected game mechanics were at the core of each interaction and iteration which gave rise to feelings of progress, competence and mastery. The final results from the DeReFrame framework and ICAD indicated that gamified approaches should be included in engineering design with CAD: in particular the game mechanics of performance feedback and rewards-achievements influence engineers’ behaviour by supporting them within the problem-solving process creating an engaging-challenging interaction. In conclusion, this research has shown that a framework, that includes both engineering requirements and gamified aspects into consideration, cam serve as a basis for implementing game-based CAD to facilitate performance by providing engaging experiences for engineers

Serious Games in Formal Education: Discussing Some Critical Aspects

International audienceInnovation in technology together with evolution in pedagogical approaches is encouraging increased integration of technology-supported interventions in mainstream teaching practices. One area attracting particularly close attention in this respect is Serious Games (SGs), which offer considerable potential for facilitating both formal and informal learning experiences in supported and standalone contexts. Advances in technology and in technology enhanced learning are raising learners' expectations for immersive and engaging game-based experiences. This trend is underpinned by the emergence of young learners adept at using digital technologies and the internet; there is an attendant risk that, as students, they may be alienated by traditional education and its failure to engage them fully in a lifelong learning process and prepare them adequately for the challenges of the 21st Century. SGs would appear to offer an attractive solution in this regard. However, there are a number of inhibitors preventing their wider take-up in mainstream education, with the result that the considerable potential on offer has yet to be fully exploited. This situation is the background for the joint efforts of partners in the Games and Learning Alliance (GALA), an EC-funded Network of Excellence on SGs, especially the sub-group dedicated to the pedagogical dimension of SGs. In its discussions on the key challenges for more wide-scale and effective SG use, the group has focused in particular on aspects related to the central role played by the educator in formal education settings. Specifically, discussion has focused on the challenges posed when educators are called on to modify their practice, adopting the new roles and approaches demanded for effective SG deployment. This paper presents the outcome of the group's exploration. It frames the question of the educator's central role by drawing on research work that, in the view of the different authors, embodies the major references for shedding light on this multi-faceted aspect. As well as the new role that the educator assumes in games-based learning environments, particular attention is also dedicated to the innovative pedagogical approaches that can be applied to SG deployment, especially those inspired by peer collaboration

Analyzing the Impact of Spatio-Temporal Sensor Resolution on Player Experience in Augmented Reality Games

Along with automating everyday tasks of human life, smartphones have become one of the most popular devices to play video games on due to their interactivity. Smartphones are embedded with various sensors which enhance their ability to adopt new new interaction techniques for video games. These integrated sen- sors, such as motion sensors or location sensors, make the device able to adopt new interaction techniques that enhance usability. However, despite their mobility and embedded sensor capacity, smartphones are limited in processing power and display area compared to desktop computer consoles. When it comes to evaluat- ing Player Experience (PX), players might not have as compelling an experience because the rich graphics environments that a desktop computer can provide are absent on a smartphone. A plausible alternative in this regard can be substituting the virtual game world with a real world game board, perceived through the device camera by rendering the digital artifacts over the camera view. This technology is widely known as Augmented Reality (AR). Smartphone sensors (e.g. GPS, accelerometer, gyro-meter, compass) have enhanced the capability for deploying Augmented Reality technology. AR has been applied to a large number of smartphone games including shooters, casual games, or puzzles. Because AR play environments are viewed through the camera, rendering the digital artifacts consistently and accurately is crucial because the digital characters need to move with respect to sensed orientation, then the accelerometer and gyroscope need to provide su ciently accurate and precise readings to make the game playable. In particular, determining the pose of the camera in space is vital as the appropriate angle to view the rendered digital characters are determined by the pose of the camera. This defines how well the players will be able interact with the digital game characters. Depending in the Quality of Service (QoS) of these sensors, the Player Experience (PX) may vary as the rendering of digital characters are affected by noisy sensors causing a loss of registration. Confronting such problem while developing AR games is di cult in general as it requires creating wide variety of game types, narratives, input modalities as well as user-testing. Moreover, current AR games developers do not have any specific guidelines for developing AR games, and concrete guidelines outlining the tradeoffs between QoS and PX for different genres and interaction techniques are required. My dissertation provides a complete view (a taxonomy) of the spatio-temporal sensor resolution depen- dency of the existing AR games. Four user experiments have been conducted and one experiment is proposed to validate the taxonomy and demonstrate the differential impact of sensor noise on gameplay of different genres of AR games in different aspect of PX. This analysis is performed in the context of a novel instru- mentation technology, which allows the controlled manipulation of QoS on position and orientation sensors. The experimental outcome demonstrated how the QoS of input sensor noise impacts the PX differently while playing AR game of different genre and the key elements creating this differential impact are - the input modality, narrative and game mechanics. Later, concrete guidelines are derived to regulate the sensor QoS as complete set of instructions to develop different genres or AR games

Data-Driven Analysis towards Monitoring Software Evolution by Continuously Understanding Changes in Users’ Needs

Ohjelmistot eivät usein vastaa käyttäjiensä odotuksia siitä huolimatta, että niiden odotetaan tarjoavan riittävä toiminnallisuus ja olevan virheettömiä. Tästä syystä ohjelmiston ylläpito on väistämätöntä ja tärkeää jokaiselle ohjelmistoyritykselle, joka haluaa pitää tuotteensa tai palvelunsa kannattavana. Koska kilpailu nykyajan ohjelmistomarkkinoilla on tiukkaa ja käyttäjien on helppo lopettaa tuotteen käyttö, yritysten on erityisen tärkeää tarkkailla ja ylläpitää käyttäjätyytyväisyyttä pitkäaikaisen menestyksen turvaamiseksi. Tämän saavuttamiseksi tärkeää on jatkuvasti ymmärtää ja kohdata käyttäjien tarpeet ja odotukset, sillä on tehokkaampaa kohdentaa ylläpito käyttäjien esittämien ongelmien perusteella. Toisaalta internet-teknologiat ovat kehittyneet nopeasti samalla, kun käyttäjien luoman sisällön määrä on kasvanut räjähdysmäisesti. Käyttäjien antama palaute (numeerinen arvostelu, ehdotus tai tekstuaalinen arvio) on esimerkki tällaisesta käyttäjien luomasta sisällöstä ja sen merkitys tuotteiden kehittämisessä asiakkaiden tarpeiden pohjalta kasvaa jatkuvasti. Käyttäjien tarpeiden ymmärtäminen on erityisen tärkeää jatkuvaa ylläpitoa ja kehitystystä vaativissa ohjelmistoissa. Tällöin on myös oleellista ymmärtää, miten asiakkaiden mielipiteet muuttuvat ajan kuluessa. Tämän lisäksi datan louhimisen ja koneoppimisen kehitys vähentävät vaivaa, joka käyttäjän tuottaman datan analysointiin ja erityisesti heidän käyttymisensä ymmärtämiseen tarvitaan. Vaikka useat tutkimukset ehdottavat tietokeskeistä lähestymistä palautteen arvioin- tiin, ohjelmiston ylläpitoa ja kehitystä hyödyntäviä lähestymistapoja on vähän. Monet menetelmät keskittyvät arvostelujen analysoinnissa tekstinlouhintaan paljastaakseen käyttäjien mielipiteet. Useat menetelmät keskittyvät myös tunnistamaan ja luokit- telemaan palautetyyppejä kuten ominaisuuspyyntöjä, virheilmoituksia ja tunteenilmauksia. Jotta ohjelmiston ylläpidosta saataisiin tehokkaampaa, tarvitaankin tehokas lähestymistapa ohjelmiston havaitun käyttäjäkokemuksen ja sen muutosten tarkkailuun ohjelmiston kehittyessä.Software products, though always being expected to provide satisfactory functionalities and be bug-free, somehow fail to meet the expectations of their users. Thus, software maintenance is inevitable and critical for any software companies who want their products or services to continue profiting. On the other hand, due to the fierce competitiveness in the contemporary software market, as well as the ease of user churns, monitoring and sustaining the satisfaction of the users is a critical criterion for the long-term success of any software products within their evolution stage. To such an end, continuously understanding and meeting the users’ needs and expectations is the key, as it is more efficient and effective to allocate maintenance effort accordingly to address the issues raised by users. On the other hand, accompanied by the rapid development of internet technologies, the volume of user-generated content has been increasing exponentially. Among such user-generated content, feedback from the customers, either numeric rating, recommendation, or textual reviews, have been playing an increasingly critical role in product designs in terms of understanding customers’ needs. Especially for software products that require constant maintenance and are continuously evolving, understanding of users’ needs and complaints, as well as the changes in their opinions through time, is of great importance. Additionally, supported by the advance of data mining and machine learning techniques, the effort of knowledge discovery from analyzing such data and specially understanding the behavior of the users shall be largely reduced. However, though many studies propose data-driven approaches for feedback analysis, the ones specifically on applying such methods supporting software maintenance and evolution are limited. Many studies focus on the text mining perspective of review analysis towards eliciting users’ opinions. Many others focus on the detection and classification of feedback types, e.g., feature requests, bug reports, and emotion expression, etc. For the purpose of enhancing the effectiveness in soft ware maintenance and evolution practice, an effective approach on the software’s perceived user experience and the monitoring of its changes during evolution is re- quired. To support the practice of software maintenance and evolution targeting enhancing user satisfaction, we propose a data-driven user review analysis approach. The contribution of this research aims to answer the following research questions: RQ1. How to analyze users’ collective expectation and perceived quality in use with data- driven approaches by exploiting sentiment and topics? RQ2. How to monitor user satisfaction over software updates during software evolution using reviews’ topics and sentiments? RQ3. How to analyze users’ profiles, software types and situational contexts as contexts of use that supports the analysis of user satisfaction? Towards answering RQ1, the thesis proposes a data-driven approach of user perceived quality evaluation and users’ needs extraction via sentiment analysis and topic modeling on large volume of user review data. Based on such outcome, the answer to RQ2 encompasses of 1) the approach to monitor user opinion changes through software evolution by detecting similar topic pairs and 2) the approach to identify the problematic updates based on anomalies in review sentiment distribution. Towards the answer to RQ3, a three-fold analysis is proposed: 1) situational contexts and ways of interaction analysis, 2) user profile and preference analysis and 3) software type and related features analysis. All the above approaches are validated by case studies. This thesis contributes to the examination of applying data-driven end user re- view analysis methods supporting software maintenance and evolution. The main implication is to enrich the existing domain knowledge of software maintenance and evolution in terms of taking advantage of the collective intelligence of end users. In addition, it conveys unique contribution to the research on software evolution con- texts in terms of various meaningful aspects and leads to a potential interdisciplinary contribution as well. On the other hand, this thesis also contributes to software maintenance and evolution practice even in the larger scope of the software industry by proposing an effective series of approaches that address critical issues within. It helps the developers ease their effort in release planning and other decision-making activities