Multimodality with Eye tracking and Haptics: A New Horizon for Serious Games?

The goal of this review is to illustrate the emerging use of multimodal virtual reality that can benefit learning-based games. The review begins with an introduction to multimodal virtual reality in serious games and we provide a brief discussion of why cognitive processes involved in learning and training are enhanced under immersive virtual environments. We initially outline studies that have used eye tracking and haptic feedback independently in serious games, and then review some innovative applications that have already combined eye tracking and haptic devices in order to provide applicable multimodal frameworks for learning-based games. Finally, some general conclusions are identified and clarified in order to advance current understanding in multimodal serious game production as well as exploring possible areas for new applications

An interface to virtual environments for people who are blind using Wii technology - mental models and navigation

Accessible games, both for serious and for entertainment purposes, would allow inclusion and participation for those with disabilities. Research into the development of accessible games, and accessible virtual environments, is discussed. Research into accessible Virtual Environments has demonstrated great potential for allowing people who are blind to explore new spaces, reducing their reliance on guides, and aiding development of more efficient spatial maps and strategies. Importantly, Lahav and Mioduser (2005, 2008) have demonstrated that, when exploring virtual spaces, people who are blind use more and different strategies than when exploring real physical spaces, and develop relatively accurate spatial representations of them. The present paper describes the design, development and evaluation of a system in which a virtual environment may be explored by people who are blind using Nintendo Wii devices, with auditory and haptic feedback. The nature of the various types of feedback is considered, with the aim of creating an intuitive and usable system. Using Wii technology has many advantages, not least of which are that it is mainstream, readily available and cheap. The potential of the system for exploration and navigation is demonstrated. Results strongly support the possibilities of the system for facilitating and supporting the construction of cognitive maps and spatial strategies. Intelligent support is discussed. Systems such as the present one will facilitate the development of accessible games, and thus enable Universal Design and accessible interactive technology to become more accepted and widespread

Affordable interactive virtual reality system for the Dynamic Hip Screw surgery training in vitro

Interactive virtual reality systems provide safe and cost-effective training environment to improve the technical skills and competence of surgeons. The trainees can have as many practice sessions, without need to the trainer all the time, before even start carrying out the procedure on any real patient. In this paper, we present an affordable interactive virtual reality system for the Dynamic Hip Screw (DHS) surgery training in vitro, through 3D tracking. The system facilitates a safe (in vitro / off patient) training to improve the cognitive coordination of trainees and junior surgeons, in particular the Hands, Eyes and Brain coordination. The system is based on very cheap commercial off-the-shelf (COT) components, which are very affordable, and needs minimum setup effort and knowledge. It also provides a range of visual and quantitative feedback information and measures, such as position, orientation, insertion point, and depth of drilling. It is envisaged that improving this level of coordination, through the training system, will contribute to reducing the failure rate of the DHS procedure. This means better treatment for patients and less costs for the Health services systems (e.g. UK's NHS system)

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Piloting mobile mixed reality simulation in paramedic distance education

New pedagogical methods delivered through mobile mixed reality (via a user-supplied mobile phone incorporating 3d printing and augmented reality) are becoming possible in distance education, shifting pedagogy from 2D images, words and videos to interactive simulations and immersive mobile skill training environments. This paper presents insights from the implementation and testing of a mobile mixed reality intervention in an Australian distance paramedic science classroom. The context of this mobile simulation study is skills acquisition in airways management focusing on direct laryngoscopy with foreign body removal. The intervention aims to assist distance education learners in practicing skills prior to attending mandatory residential schools and helps build a baseline equality between those students that study face to face and those at a distance. Outcomes from the pilot study showed improvements in several key performance indicators in the distance learners, but also demonstrated problems to overcome in the pedagogical method

Dementia, music and biometric gaming: Rising to the Dementia Challenge

In 2012, the U.K. government launched its Dementia Challenge, authorizing additional funding for dementia research and health care. The search for curative medicines is ongoing, but scientific research reveals evidence that music can play a positive role in general health, and in dementia and Alzheimer’s disease in particular. This article considers whether some of the challenges that dementia presents could be addressed through music therapy and proposes that biometric gaming might offer one means of channeling such associated health benefits to sufferers of dementia, even in the final stages of the disease

A survey of haptics in serious gaming

Serious gaming often requires high level of realism for training and learning purposes. Haptic technology has been proved to be useful in many applications with an additional perception modality complementary to the audio and the vision. It provides novel user experience to enhance the immersion of virtual reality with a physical control-layer. This survey focuses on the haptic technology and its applications in serious gaming. Several categories of related applications are listed and discussed in details, primarily on haptics acts as cognitive aux and main component in serious games design. We categorize haptic devices into tactile, force feedback and hybrid ones to suit different haptic interfaces, followed by description of common haptic gadgets in gaming. Haptic modeling methods, in particular, available SDKs or libraries either for commercial or academic usage, are summarized. We also analyze the existing research difficulties and technology bottleneck with haptics and foresee the future research directions