Serious Game Evaluation as a Meta-game

Purpose – This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach – In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team. Findings – The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in-role game experience. Practical implications – This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user. Originality/value – Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in-game experience that added value to the user experience whilst also yielding relevant results for the development team

User Experience Evaluation in BCI: Filling the Gap

Brain-computer interface (BCI) systems can improve the user experience (UX) when used in entertainment technologies. Improved UX can enhance user acceptance, improve quality of life and also increase the system performance of a BCI system. Therefore, the evaluation of UX is essential in BCI research. However, BCI systems are generally evaluated according to the system aspect only so there is no methodology to evaluate UX in BCI systems. This paper gives an overview of such methods from the human-computer interaction field and discusses their possible uses in BCI research

User Experience Evaluation in BCI:Mind the Gap!

Generally brain-computer interface (BCI) systems are evaluated based on the assumption that the user is trying to perform a specific task in the most efficient way. BCI for entertainment yields interesting applications for both patients and healthy users. Then the purpose is to create positive experiences that enrich our lives. To evaluate such systems, the user experience needs to be taken into account to understand how a system can satisfy these needs. This paper points at the gap in user experience evaluation currently in BCI research, and shows how user experience evaluation could benefit BCI, through increased user acceptance, enjoyment, BCI task performance, enhanced human-computer interaction, and improved selection of suitable mental tasks in a given context

User expectations and experiences of a speech and thought controlled computer game

Brain-computer interfaces (BCIs) are often evaluated in terms of performance and seldom for usability. However in some application domains, such as entertainment computing, user experience evaluation is vital. User experience evaluation in BCI systems, especially in entertainment applications such as games, can be biased due to the novelty of the interface. However, as the novelty will eventually vanish, what matters is the user experience related to the unique features offered by BCI. Therefore it is a viable approach to compare BCI to other novel modalities, such as a speech or motion recogniser, rather than the traditional mouse and keyboard. In the study which we present in this paper, our participants played a computer game with a BCI and an automatic speech recogniser (ASR) and they rated their expectations and experiences for both modalities. Our analysis on subjective ratings revealed that both ASR and BCI were successful in satisfying participants' expectations in general. Participants found speech control easier to learn than BCI control. They indicated that BCI control induced more fatigue than they expected

Meeting the expectations from brain-computer interfaces

Brain-computer interfaces (BCIs) are often evaluated in terms of performance and seldom for usability. However in some application domains, such as entertainment computing, user experience evaluation is vital. User experience evaluation in BCI systems, especially in entertainment applications such as games, can be biased due to the novelty of the interface.However, as the novelty will eventually vanish, whatmatters is the user experience related to the unique features offered by BCI. Therefore it is a viable approach to compare BCI to other novel modalities, such as a speech or motion recognizer, rather than the traditional mouse and keyboard. In the study that we present in this article, our participants played a computer game with a BCI and an automatic speech recognizer (ASR), and they rated their expectations and experiences for both modalities. Our analysis on subjective ratings revealed that both ASR and BCI were successful in satisfying participants' expectations in general. Participants found speech control easier to learn than BCI control. They indicated that BCI control induced more fatigue than they expected

User Experience of Interactive Omnidirectional Video: Case Hugo Simberg VR

Virtual reality technology is gaining popularity and market saturation with new types of head- mounted display devices reaching the market in the recent years. With more conventional uses, such as industrial simulators for flight lessons, the newer technologies have found their way on the consumer market, being used at home for entertainment and gaming. An emerging way to create these types of applications is using omnidirectional video as the basis. These Interactive omnidirectional video applications can be developed comparatively quickly, while still offering some challenges in the development process. User Experience Design in the field of virtual reality has been gaining more foothold in the field of virtual reality application development. User experience offers a multitude of design frameworks and approaches to find the user needs and to ensure for the best user experience. While this design process is suitable for development on this field, the user experience and design implications regarding virtual reality and interactive omnidirectional remain quite limited. User Experience design methods were used in a development process for an interactive omnidirectional video application, Hugo Simberg VR, which allows the user to explore a museum and a cathedral virtually. This thesis discusses this undertaken design process, from the initial exploratory design phases to the iterative refining design phases, and the evaluation of the finished application. The results of the evaluation identified nine experiential dimensions in interactive omnidirectional video, of which six were perceived with positive valence, and three with negative valence. These findings show that these dimensions should be considered in the design of interactive omnidirectional videos

Heuristic Evaluation for Serious Immersive Games and M-instruction

© Springer International Publishing Switzerland 2016. Two fast growing areas for technology-enhanced learning are serious games and mobile instruction (M-instruction or M-Learning). Serious games are ones that are meant to be more than just entertainment. They have a serious use to educate or promote other types of activity. Immersive Games frequently involve many players interacting in a shared rich and complex-perhaps web-based-mixed reality world, where their circumstances will be multi and varied. Their reality may be augmented and often self-composed, as in a user-defined avatar in a virtual world. M-instruction and M-Learning is learning on the move; much of modern computer use is via smart devices, pads, and laptops. People use these devices all over the place and thus it is a natural extension to want to use these devices where they are to learn. This presents a problem if we wish to evaluate the effectiveness of the pedagogic media they are using. We have no way of knowing their situation, circumstance, education background and motivation, or potentially of the customisation of the final software they are using. Getting to the end user itself may also be problematic; these are learning environments that people will dip into at opportune moments. If access to the end user is hard because of location and user self-personalisation, then one solution is to look at the software before it goes out. Heuristic Evaluation allows us to get User Interface (UI) and User Experience (UX) experts to reflect on the software before it is deployed. The effective use of heuristic evaluation with pedagogical software [1] is extended here, with existing Heuristics Evaluation Methods that make the technique applicable to Serious Immersive Games and mobile instruction (M-instruction). We also consider how existing Heuristic Methods may be adopted. The result represents a new way of making this methodology applicable to this new developing area of learning technology

Biometric storyboards: a games user research approach for improving qualitative evaluations of player experience

Developing video games is an iterative and demanding process. It is difficult to achieve the goal of most video games — to be enjoyable, engaging and to create revenue for game developers — because of many hard-to-evaluate factors, such as the different ways players can interact with the game. Understanding how players behave during gameplay is of vital importance to developers and can be uncovered in user tests as part of game development. This can help developers to identify and resolve any potential problem areas before release, leading to a better player experience and possibly higher game review scores and sales. However, traditional user testing methods were developed for function and efficiency oriented applications. Hence, many traditional user testing methods cannot be applied in the same way for video game evaluation. This thesis presents an investigation into the contributions of physiological measurements in user testing within games user research (GUR). GUR specifically studies the interaction between a game and users (players) with the aim to provide feedback for developers to help them to optimise the game design of their title. An evaluation technique called Biometric Storyboards is developed, which visualises the relationships between game events, player feedback and changes in a player’s physiological state. Biometric Storyboards contributes to the field of human-computer interaction and GUR in three important areas: (1) visualising mixedmeasures of player experience, (2) deconstructing game design by analysing game events and pace, (3) incremental improvement of classic user research techniques (such as interviews and physiological measurements). These contributions are described in practical case studies, interviews with game developers and laboratory experiments. The results show this evaluation approach can enable games user researchers to increase the plausibility and persuasiveness of their reports and facilitate developers to better deliver their design goals. Biometric Storyboards is not aimed at replacing existing methods, but to extend them with mixed methods visualisations, to provide powerful tools for games user researchers and developers to better understand and communicate player needs, interactions and experiences. The contributions of this thesis are directly applicable for user researchers and game developers, as well as for researchers in user experience evaluation in entertainment systems

Modeling Online to Offline E-Business User Experience

Complex business patterns from providers and diverse needs of customers in E-commerce necessitate constant and rapid modifications of services for new business models. The online-to-offline (O2O) business model is one of newly developed models where smartphones, tablets, e-readers and digital signages provide mobility and entertainment that ordinary people have never experienced before. Business opportunities are brought forth by O2O from cyberspace into daily reality. User experience (UX) evaluation is thus getting more and more important for user-centered design. In the literature, UX evaluation methodologies were designed for usability and satisfaction of computers or consumer electronics. Their results, however, did not necessarily help design O2O smart business services in that UX constructs in O2O should differ from that in using computers or consumer electronics. Important constructs of UX in O2O such as usability, utility, adoptability and desirability should be considered in the design of service flows and activities. The current study proposed important UX constructs for assessing O2O design by analyzing O2O users. A rapid evaluation method for UX in O2O can be developed based on the constructs

Will mobile video become the killer application for 3G? - an empirical model for media convergence

Mobile carriers have continually rolled out 3G mobile video applications to increase their revenue and profits. The presumption is that video is superior to the already successful SMS, ringtones, and pictures, and can create greater value to users. However, recent market surveys revealed contradicting results. Motivated by this discrepancy, we propose in this paper a parsimonious model for user acceptance of mobile entertainment as digital convergence. Integrating research on Information Systems, Flow, and Media Psychology, we take a unique approach to user acceptance of digital convergence - platform migration. Our key proposition is that the interaction between media types and the platform-specific constraints is the key determinant of user evaluation. Particularly, users' involvement in the media is determined by both the entertaining time span on the original platform and the attentional constraint of the new platform. The mismatch between the two spans can result in lower level involvement, which in turn cause no or even negative user emotional responses. The model was tested with empirical data. We discuss the theoretical contributions, strategic and design implications, and future research directions derived from this theoretical framewor