The design-by-adaptation approach to universal access: learning from videogame technology

This paper proposes an alternative approach to the design of universally accessible interfaces to that provided by formal design frameworks applied ab initio to the development of new software. This approach, design-byadaptation, involves the transfer of interface technology and/or design principles from one application domain to another, in situations where the recipient domain is similar to the host domain in terms of modelled systems, tasks and users. Using the example of interaction in 3D virtual environments, the paper explores how principles underlying the design of videogame interfaces may be applied to a broad family of visualization and analysis software which handles geographical data (virtual geographic environments, or VGEs). One of the motivations behind the current study is that VGE technology lags some way behind videogame technology in the modelling of 3D environments, and has a less-developed track record in providing the variety of interaction methods needed to undertake varied tasks in 3D virtual worlds by users with varied levels of experience. The current analysis extracted a set of interaction principles from videogames which were used to devise a set of 3D task interfaces that have been implemented in a prototype VGE for formal evaluation

Analyzing the Impact of Spatio-Temporal Sensor Resolution on Player Experience in Augmented Reality Games

Along with automating everyday tasks of human life, smartphones have become one of the most popular devices to play video games on due to their interactivity. Smartphones are embedded with various sensors which enhance their ability to adopt new new interaction techniques for video games. These integrated sen- sors, such as motion sensors or location sensors, make the device able to adopt new interaction techniques that enhance usability. However, despite their mobility and embedded sensor capacity, smartphones are limited in processing power and display area compared to desktop computer consoles. When it comes to evaluat- ing Player Experience (PX), players might not have as compelling an experience because the rich graphics environments that a desktop computer can provide are absent on a smartphone. A plausible alternative in this regard can be substituting the virtual game world with a real world game board, perceived through the device camera by rendering the digital artifacts over the camera view. This technology is widely known as Augmented Reality (AR). Smartphone sensors (e.g. GPS, accelerometer, gyro-meter, compass) have enhanced the capability for deploying Augmented Reality technology. AR has been applied to a large number of smartphone games including shooters, casual games, or puzzles. Because AR play environments are viewed through the camera, rendering the digital artifacts consistently and accurately is crucial because the digital characters need to move with respect to sensed orientation, then the accelerometer and gyroscope need to provide su ciently accurate and precise readings to make the game playable. In particular, determining the pose of the camera in space is vital as the appropriate angle to view the rendered digital characters are determined by the pose of the camera. This defines how well the players will be able interact with the digital game characters. Depending in the Quality of Service (QoS) of these sensors, the Player Experience (PX) may vary as the rendering of digital characters are affected by noisy sensors causing a loss of registration. Confronting such problem while developing AR games is di cult in general as it requires creating wide variety of game types, narratives, input modalities as well as user-testing. Moreover, current AR games developers do not have any specific guidelines for developing AR games, and concrete guidelines outlining the tradeoffs between QoS and PX for different genres and interaction techniques are required. My dissertation provides a complete view (a taxonomy) of the spatio-temporal sensor resolution depen- dency of the existing AR games. Four user experiments have been conducted and one experiment is proposed to validate the taxonomy and demonstrate the differential impact of sensor noise on gameplay of different genres of AR games in different aspect of PX. This analysis is performed in the context of a novel instru- mentation technology, which allows the controlled manipulation of QoS on position and orientation sensors. The experimental outcome demonstrated how the QoS of input sensor noise impacts the PX differently while playing AR game of different genre and the key elements creating this differential impact are - the input modality, narrative and game mechanics. Later, concrete guidelines are derived to regulate the sensor QoS as complete set of instructions to develop different genres or AR games

It’s not the model that doesn’t fit, it’s the controller! The role of cognitive skills in understanding the links between natural mapping, performance, and enjoyment of console video games

This study examines differences in performance, frustration, and game ratings of individuals playing first person shooter video games using two different controllers (motion controller and a traditional, pushbutton controller) in a within-subjects, randomized order design. Structural equation modeling was used to demonstrate that cognitive skills such as mental rotation ability and eye/hand coordination predicted performance for both controllers, but the motion control was significantly more frustrating. Moreover, increased performance was only related to game ratings for the traditional controller input. We interpret these data as evidence that, contrary to the assumption that motion controlled interfaces are more naturally mapped than traditional push-button controllers, the traditional controller was more naturally mapped as an interface for gameplay

Characterizing the Effects of Local Latency on Aim Performance in First Person Shooters

Real-time games such as first-person shooters (FPS) are sensitive to even small amounts of lag. The effects of network latency have been studied, but less is known about local latency -- that is, the lag caused by local sources such as input devices, displays, and the application. While local latency is important to gamers, we do not know how it affects aiming performance and whether we can reduce its negative effects. To explore these issues, we tested local latency in a variety of real-world gaming systems and carried out a controlled study focusing on targeting and tracking activities in an FPS game with varying degrees of local latency. In addition, we tested the ability of a lag compensation technique (based on aim assistance) to mitigate the negative effects. To motivate the need for these studies, we also examined how aim in FPS differs from pointing in standard 2D tasks, showing significant differences in performance metrics. Our studies found local latencies in the real-world range from 23 to 243~ms that cause significant and substantial degradation in performance (even for latencies as low as 41~ms). The studies also showed that our compensation technique worked well, reducing the problems caused by lag in the case of targeting, and removing the problem altogether in the case of tracking. Our work shows that local latency is a real and substantial problem -- but game developers can mitigate the problem with appropriate compensation methods

The design-by-adaptation approach to universal access: learning from videogame technology

This paper proposes an alternative approach to the design of universally accessible interfaces to that provided by formal design frameworks applied ab initio to the development of new software. This approach, design-byadaptation, involves the transfer of interface technology and/or design principles from one application domain to another, in situations where the recipient domain is similar to the host domain in terms of modelled systems, tasks and users. Using the example of interaction in 3D virtual environments, the paper explores how principles underlying the design of videogame interfaces may be applied to a broad family of visualization and analysis software which handles geographical data (virtual geographic environments, or VGEs). One of the motivations behind the current study is that VGE technology lags some way behind videogame technology in the modelling of 3D environments, and has a less-developed track record in providing the variety of interaction methods needed to undertake varied tasks in 3D virtual worlds by users with varied levels of experience. The current analysis extracted a set of interaction principles from videogames which were used to devise a set of 3D task interfaces that have been implemented in a prototype VGE for formal evaluation

Foveated Video Streaming for Cloud Gaming

Good user experience with interactive cloud-based multimedia applications, such as cloud gaming and cloud-based VR, requires low end-to-end latency and large amounts of downstream network bandwidth at the same time. In this paper, we present a foveated video streaming system for cloud gaming. The system adapts video stream quality by adjusting the encoding parameters on the fly to match the player's gaze position. We conduct measurements with a prototype that we developed for a cloud gaming system in conjunction with eye tracker hardware. Evaluation results suggest that such foveated streaming can reduce bandwidth requirements by even more than 50% depending on parametrization of the foveated video coding and that it is feasible from the latency perspective.Comment: Submitted to: IEEE 19th International Workshop on Multimedia Signal Processin

Techno-historical limits of the interface: the performance of interactive narrative experiences

This thesis takes the position that current analyses of digitally mediated interactive experiences that include narrative elements often lack adequate consideration of the technical and historical contexts of their production.From this position, this thesis asks the question: how is the reader/player/user's participation in interactive narrative experiences (such as hypertext fiction, interactive fiction, computer games, and electronic art) influenced by the technical and historical limitations of the interface?In order to investigate this question, this thesis develops a single methodology from relevant media and narrative theory, in order to facilitate a comparative analysis of well known exemplars from distinct categories of digitally mediated experiences. These exemplars are the interactive fiction Adventure, the interactive art work Osmose, the hypertext fiction Afternoon, a story, and the computer/video games Myst, Doom, Half Life and Everquest.The main argument of this thesis is that the technical limits of new media experiences cause significant ‘gaps’ in the reader’s experience of them, and that the cause of these gaps is the lack of a dedicated technology for new media, which instead ‘borrows’ technology from other fields. These gaps are overcome by a greater dependence upon the reader’s cognitive abilities than other media forms. This greater dependence can be described as a ‘performance’ by the reader/player/user, utilising Eco’s definition of an ‘open’ work (Eco 21).This thesis further argues that the ‘mimetic’ and ‘immersive’ ambitions of current new media practice can increases these gaps, rather than overcoming them. The thesis also presents the case that these ‘gaps’ are often not caused by technical limits in the present, but are oversights by the author/designers that have arisen as the product of a craft culture that has been subject to significant technical limitations in the past. Compromises that originally existed to overcome technical limits have become conventions of the reader/player/user’s interactive literacy, even though these conventions impinge on the experience, and are no longer necessary because of subsequent technical advances. As a result, current new media users and designers now think of these limitations as natural.This thesis concludes the argument by redefining ‘immersion’ as the investment the reader makes to overcome the gaps in an experience, and suggests that this investment is an important aspect of their performance of the work