Gaze-directed gameplay in first person computer games

The use of eye tracking systems in computer games is still at an early stage. Commercial eye trackers and researches have been focusing in gaze-oriented gameplay as an alternative to traditional input devices. This dissertation proposes to investigate the advantages and disadvantages of the use of these systems in computer games. For it, instead of using eye tracking as a simple direct control input, it is proposed to use it in order to control the attention of the player’s avatar (e.g., if the player notices an obstacle in the way, the avatar will notice it too and avoid it) and the game’s procedural content generation (e.g., spawn obstacles in the opposite side of the screen to where the player’s attention is focused). To demonstrate the value of this proposal, it was developed and is herein presented the first-person shooter "Zombie Runner". Tests showed that the implementation meets the stipulated technical requirements and that, although it still needs improvements in terms of precision and robustness, eye tracking technology can be successfully used to to make the player experience more immersive and challenging.A utilização de sistemas de rastreamento ocular em jogos de computador ainda se encontra numa fase embrionária. Aparelhos de rastreamento ocular comerciais e pesquisas na área têm-se focado em jogabilidade à base da atenção visual como uma alternativa a métodos de entrada tradicionais. Esta dissertação propõe-se a investigar as vantages e desvantagens do uso destes sistemas em jogos de computador. Para isso, invés de se usar rastreamento ocular apenas como um método directo de entrada, é proposto usá-lo para controlar a atenção do personagem do jogo (e.g., se o jogador reparar num obstáculo, a personagem também repara e desvia-se do mesmo) assim como afectar a geração procedimental do jogo (e.g., gerar obstáculos no lado oposto ao qual o jogador tem a sua atenção focada). Para demonstrar o valor desta proposta, foi desenvolvido e aqui apresentado o jogo de tiros em primeira pessoa "Zombie Runner". Os testes demonstraram que a implementação cumpre os requisitos técnicos estipulados e que, apesar de ainda carecer de melhorias em termos de precisão e robustez, a tecnologia para rastreamento ocular pode ser utilizada com sucesso para tornar a experiência do jogador mais imersiva e desafiante

Preferences, emotions, and visual attention in the first-person shooter game experience

First-person shooter (FPS) games are one of the most popular yet notorious genres of digital games. They contain visceral emotional content and require intense visual attention from players, leading some people to appreciate and others to resent these types of games. This thesis investigated individual differences in the game experience of FPS games by exploring how preferences for violent game dynamics (e.g., preferences for shooting, killing, and exploding) affect players’ emotions and perceptions of curiosity, vitality, and self-efficacy. In addition, the thesis explored how visual attention skills affect the viewing of FPS games as indexed by viewers’ eye movements. In Study I, the role of visual attention skills in viewing FPS gameplay videos was explored. The results showed that viewers’ eye movements tended to progress from a diffuse scanning mode towards a more focal and central viewing mode as time passed. Visual qualities and saliency of events also guided eye movements. Individual differences in visual attention skills (namely, the ability to track multiple objects, perform a visual search for targets, and to see rapidly appearing serial targets) were related to what was attended to in the screen. The role of visual attention skills on eye movements was more prominent during visually distinct events. In sum, the results showed that specific visual attention skills predicted eye movement patterns during FPS gameplay video viewing. Study II explored whether game dynamics preferences and emotion-related responses to an FPS game are connected. Participants’ heart rate, electrodermal activity, and electric activity of facial muscles were recorded as indexes of emotion-related responses both during playing (active participation) and gameplay video viewing (passive watching). The participants also rated their level of experienced arousal and valence. The results showed that there were individual differences in physiological emotion-related responses as a function of dynamics preferences, especially in measures of physiological arousal. Those who liked violent dynamics showed a rather stable level of physiological arousal state both when playing and when viewing the game. In contrast, participants who disliked violent dynamics showed an overall higher level of physiological arousal during playing than when viewing, and the level of arousal increased across time in both conditions. The results on facial muscle activity likewise showed that activity differed between people who liked versus disliked violent dynamics. However, the results were somewhat conflicting: those who liked violent game dynamics showed a steep increase in the activity of the corrugator supercilii muscle, an index of negative valence. Instead, those who disliked the dynamics showed less increase in corrugator supercilii activity. The dynamics preferences did not affect self-reported emotional valence or arousal. Thus, the results highlight that game dynamics preferences were associated with physiological signals, although they may not be a straightforward index of emotions in a gaming context. In Study III, associations between game dynamics preferences and self-reported experiences of vitality, self-efficacy, and curiosity were explored both in association with life in general and with playing an FPS video game. The results showed that players who were neutral or mildly positive towards violent content experienced stable levels of vitality, curiosity, and emotional valence both in life in general and when playing. They also experienced a slight decline in self-efficacy in the playing context. Conversely, those who disliked violent dynamics experienced a clear decline in all of these measures in the playing context. Thus, game dynamics preferences were connected with wider experiential reflections related to playing. Overall, the results of all three studies showed why there is individual variation in the playing experience: players and viewers have differing skills and preferences. These skills and preferences affect how players and viewers pay attention to the game, and what kind of emotional reactions and experiences they have. This is valuable for understanding the psychological outcomes of FPS games, as well as why people hold differing opinions about these types of games. Likewise, the results have importance for game design, as they show that players respond in different ways to game contents. Thus, it may be fruitful to personalize and tailor game contents based on players’ preferences and visual attention skills.Mieltymykset, tunteet ja näönvarainen tarkkaavuus ensimmäisen persoonan ammuntapelien pelikokemuksessa Ensimmäisen persoonan ammuntapelit (FPS-pelit) ovat sekä yksi suosituimmista että pahamaineisimmista digitaalisten pelien genreistä. Ne sisältävät voimakasta tunnepitoista sisältöä ja vaativat äärimmäistä näönvaraista tarkkaavuutta. Näiden seikkojen takia toiset arvostavat ja toiset paheksuvat kyseisiä pelejä. Tässä väitöskirjassa tutkittiin yksilöllisiä eroja FPS-pelien pelikokemuksessa selvittämällä, kuinka mieltymykset väkivaltaisiin pelidynamiikkoihin (esimerkiksi mieltymykset ampumiseen, tappamiseen ja räjäyttämiseen) vaikuttavat pelaajien tunnetiloihin ja kokemuksiin uteliaisuudesta, elinvoimaisuudesta ja minäpystyvyydestä. Tämän lisäksi väitöskirjassa tutkittiin silmänliikkeitä tarkastelemalla kuinka näönvaraisen tarkkaavuuden taidot ovat yhteydessä FPS-pelivideoiden katseluun. Tutkimuksessa I tarkasteltiin, miten näönvaraisen tarkkaavuuden taidot vaikuttavat FPS-pelivideoiden katseluun. Tutkimuksessa havaittiin, että silmänliikkeet etenivät laajemmasta ja hajaantuneemmasta silmäilystä kohti pinta-alaltaan pienempää ja lähempänä näytön keskustaa olevaa aluetta. Lisäksi erilaisten pelitapahtumien visuaaliset ominaisuudet ja huomiota herättävyys suuntasivat silmänliikkeitä. Yksilölliset erot näönvaraisen tarkkaavuuden taidoissa (tässä tutkimuksessa taidot seurata useita liikkuvia kohteita, etsiä kohteita ja nähdä nopeasti peräkkäin ilmestyviä kohteita) olivat yhteydessä siihen, miten katselija tarkasteli pelinäkymää. Näönvaraisen tarkkaavuuden taitojen yhteys silmänliikkeisiin tuli esiin etenkin visuaalisesti toisistaan erottuvien pelitapahtumien aikana. Yhteenvetona tutkimuksen tulokset osoittivat, että tietyt näönvaraisen tarkkaavuuden taidot ennustavat silmänliikkeitä FPS-pelivideon katselun aikana. Tutkimuksessa II selvitettiin, ovatko pelidynamiikkamieltymykset ja FPS-peliin liittyvät tunnereaktiot yhteydessä toisiinsa. Tutkimukseen osallistujien sykettä, ihon sähkönjohtavuutta ja kasvolihasten sähköistä aktiivisuutta mitattiin indikaatioina tunnereaktioista sekä pelaamisen (aktiivinen osallistuminen) että pelivideon katselun (passiivinen tarkkailu) aikana. Osallistujat myös arvioivat oman tunnetilansa koettua virittävyyttä ja valenssia. Tulokset osoittivat, että yksilöiden välillä oli eroja fysiologisissa tunteisiin liittyvissä reaktioissa riippuen siitä, millaiset pelidynamiikkamieltymykset heillä oli. Tämä näkyi erityisesti fysiologisissa autonomisen hermoston tilaa kuvaavissa mittareissa. Väkivaltaisista dynamiikoista pitävillä osallistujilla oli suhteellisen tasainen fysiologisen aktivaation taso sekä pelatessa että pelivideota katsellessa. Sen sijaan niillä osallistujilla jotka eivät pitäneet väkivaltaisista dynamiikoista oli kaiken kaikkiaan korkeampi fysiologisen aktivaation taso pelatessa kuin pelivideota katsellessa, ja aktivaation taso kasvoi ajan kuluessa molemmissa tilanteissa. Kasvojen lihasten sähköiseen toimintaan liittyvät tulokset niin ikään osoittivat, että väkivaltaisista dynamiikoista pitävien ja niitä vieroksuvien henkilöiden välillä oli eroja. Tulokset olivat kuitenkin jossain määrin ristiriitaisia: väkivaltaisista dynamiikoista pitävillä osallistujilla negatiivista valenssia indikoiva corrugator supercilii -lihaksen aktiivisuus lisääntyi ajan kuluessa huomattavasti. Sen sijaan osallistujilla jotka eivät pitäneet väkivaltaisista dynamiikoista corrugator supercilii -lihaksen aktiivisuuden lisääntyminen oli lievempää. Pelidynamiikkamieltymykset eivät olleet yhteydessä osallistujien omiin arvioihin tunnekokemuksen virittävyydestä ja valenssista. Täten tulokset osoittivat, että pelidynamiikkamieltymykset olivat yhteydessä fysiologisiin vasteisiin, mutta niitä ei voida käyttää täysin mutkattomina mittareina tunteista pelikontekstissa. Tutkimuksessa III tarkasteltiin pelidynamiikkamieltymysten yhteyttä uteliaisuuden, elinvoimaisuuden ja minäpystyvyyden kokemuksiin elämässä ylipäätään ja FPS-pelin pelaamiseen liittyen. Tutkimuksessa havaittiin, että uteliaisuus, elinvoimaisuus ja tunteen valenssi olivat samankaltaiset sekä elämässä ylipäätään että pelatessa mikäli pelaaja suhtautui väkivaltaisiin dynamiikkoihin neutraalisti tai jonkin verran positiivisesti. Tällaisilla pelaajilla minäpystyvyys oli kuitenkin jonkin verran alhaisempi pelitilanteessa verrattuna elämään ylipäätään. Sen sijaan pelaajat jotka eivät pitäneet väkivaltaisista dynamiikoista arvioivat kaikkien näiden kokemusten olevan selvästi huonompia pelatessa. Pelidynamiikkamieltymykset olivat siis yhteydessä laajempiin reflektiivisiin kokemuksiin pelaamisesta. Väitöstutkimuksen tulokset auttavat ymmärtämään, miksi pelikokemuksessa on yksilöllistä vaihtelua: pelaajat ja katsojat eroavat taidoiltaan ja mieltymyksiltään. Nämä taidot ja mieltymykset ovat yhteydessä siihen, millä tavoin pelaajat ja katselijat kiinnittävät huomiota peliin ja minkälaisia tunnereaktioita ja kokemuksia heillä on. Näiden seikkojen huomioiminen on tärkeää FPS-pelien psykologisten vaikutusten ja peleihin liittyvien eriävien mielipiteiden ymmärtämiseksi. Tuloksilla on lisäksi merkitystä pelisuunnittelulle, sillä ne osoittavat, että pelaajat reagoivat eri tavoin pelisisältöihin. Tämän vuoksi voisi olla hyödyllistä muovata pelisisältöjä yksilöllisesti pelaajien mieltymysten ja näönvaraisen tarkkaavuuden taitojen mukaan

Foveated Video Streaming for Cloud Gaming

Video gaming is generally a computationally intensive application and to provide a pleasant user experience specialized hardware like Graphic Processing Units may be required. Computational resources and power consumption are constraints which limit visually complex gaming on, for example, laptops, tablets and smart phones. Cloud gaming may be a possible approach towards providing a pleasant gaming experience on thin clients which have limited computational and energy resources. In a cloud gaming architecture, the game-play video is rendered and encoded in the cloud and streamed to a client where it is displayed. User inputs are captured at the client and streamed back to the server, where they are relayed to the game. High quality of experience requires the streamed video to be of high visual quality which translates to substantial downstream bandwidth requirements. The visual perception of the human eye is non-uniform, being maximum along the optical axis of the eye and dropping off rapidly away from it. This phenomenon, called foveation, makes the practice of encoding all areas of a video frame with the same resolution wasteful. In this thesis, foveated video streaming from a cloud gaming server to a cloud gaming client is investigated. A prototype cloud gaming system with foveated video streaming is implemented. The cloud gaming server of the prototype is configured to encode gameplay video in a foveated fashion based on gaze location data provided by the cloud gaming client. The effect of foveated encoding on the output bitrate of the streamed video is investigated. Measurements are performed using games from various genres and with different player points of view to explore changes in video bitrate with different parameters of foveation. Latencies involved in foveated video streaming for cloud gaming, including latency of the eye tracker used in the thesis, are also briefly discussed

Games technology: console architectures, game engines and invisible interaction

This presentation will look at three core developments in games technology. First we will look at the architectural foundations on which the consoles are built to deliver games performance. Millions of consoles are sold and the console performance is improving in parallel. Next we look at the cutting edge features available in game engines. Middleware software, namely game engines, help developers build games with rich features and also simultaneously harness the power of the game consoles to satisfy gamers. The third part focuses on Invisible Game Interaction. The Nintendo Wii games console was an instant success because of the Wiimote. Old and young alike embraced it. The Microsoft Kinect pushed the boundary even further, where the interaction device is slowly becoming invisible and the human body becomes the interface. Finally, we look at novel research developments that go beyond current game interaction devices

Eye Movements during dynamic scene viewing are affected by visual attention skills and events of the scene: Evidence from first-person shooter gameplay videos

The role of individual differences during dynamic scene viewing was explored. Participants (N=38) watched a gameplay video of a first-person shooter (FPS) videogame while their eye movements were recorded. In addition, the participants’ skills in three visual attention tasks (attentional blink, visual search, and multiple object tracking) were assessed.  The results showed that individual differences in visual attention tasks were associated with eye movement patterns observed during viewing of the gameplay video. The differences were noted in four eye movement measures: number of fixations, fixation durations, saccade amplitudes and fixation distances from the center of the screen. The individual differences showed during specific events of the video as well as during the video as a whole. The results highlight that an unedited, fast-paced and cluttered dynamic scene can bring about individual differences in dynamic scene viewing

Visual Attention in Dynamic Environments and its Application to Playing Online Games

Abstract In this thesis we present a prototype of Cognitive Programs (CPs) - an executive controller built on top of Selective Tuning (ST) model of attention. CPs enable top-down control of visual system and interaction between the low-level vision and higher-level task demands. Abstract We implement a subset of CPs for playing online video games in real time using only visual input. Two commercial closed-source games - Canabalt and Robot Unicorn Attack - are used for evaluation. Their simple gameplay and minimal controls put the emphasis on reaction speed and attention over planning. Abstract Our implementation of Cognitive Programs plays both games at human expert level, which experimentally proves the validity of the concept. Additionally we resolved multiple theoretical and engineering issues, e.g. extending the CPs to dynamic environments, finding suitable data structures for describing the task and information flow within the network and determining the correct timing for each process

Comparison of EEG biofeedback and visual search strategies during e-sports play according to skill level

About 83% of the perceptual information humans receive from the outside world comes through the visual domain, making the tracking of visual information important for the superior performance of e-sports players. In addition, it is known that e-sports players’ emotional state during performance affects their results. However, few studies have examined visual search strategies and electroencephalogram (EEG) findings of e-sports players while engaged in e-sports. Therefore, the present study aimed to investigate the characteristics of visual search activity and EEG feedback during first person shooter (FPS) game play, in which the tracking of visual information is important, and to examine the characteristics of different gaming skill levels. Four skilled and five semi-skilled e-sports participants (mean age ± 19.11, SD = 0.99) participated in this study. Gazepoint GP3 (Canada) was used to measure eye movements, and the Sports KANSEI (Littlesoftware Inc., Japan) was used to analyse the emotional state of the participants by using EEG data. The results showed that the skilled e-sports players checked the camera significantly more frequently than the semi-skilled participants. The results of the area of interest (AOI) analysis showed that skilled e-sports players glanced at the friendly team information displayed at the top of the game screen more often than semi-skilled players. Furthermore, it was found that skilled e-sports players had significantly higher average EEG relaxation values during gameplay than semi-skilled players. Thus, the visual search strategy and emotional state while playing e-sports differed according to gaming skill level

Through Her Eyes: The Gendering of Female First-Person Shooters

While the video game industry has attempted to address their years of mistreatment towards women, within games and how they are produced, by hiring more women and including more female characters as playable options, these fixes have been superficial at best. Not only are there still few females as main characters in video games, but that there are so few female video games. By this I refer to the fact that video games told through the eyes of female characters often do not feature a gendered narrative, unlike multiple games with male POVs in which the storyline directly reflects their gender. This issue, however, is not just about inclusion of more female stories, but also execution. Female FPSs may lack a narrative reflecting their gender, but they often feature gameplay that represents a stereotype of females as weaker and less aggressive than men. The purpose of this analysis is to explore how first-person shooter video games gender (or not) their female texts, through both narrative and gameplay

Immunitary Gaming: Mapping the First-Person Shooter

Videogames have been theorised as an action-based medium. The original contribution to knowledge this thesis makes is to reconfigure this claim by considering popular multiplayer FPS games as reaction-based – particularly, immune reactions. I take up Roberto Esposito’s claim that the individual in contemporary biopolitics is defined negatively against the other, controlled and ultimately negated via their reactions to power’s capacity to incessantly generate threats. By inciting insecurity and self-protective gestures, FPS games like Activision’s Call of Duty franchise and EA’s Battlefield series vividly dramatise Esposito’s thought, producing an immunitary gaming. Immunitary Gaming locates the FPS within key moments of change as well as evolution in Western image systems including the emergence of linear perspective, cartography and the early years of the cinema. The FPS appropriates these image systems, but also alters their politics. Giorgio Agamben has argued that the apparatuses of late modernity no longer subjectify like their forebears, but desubjectify the individual, producing an impotent neoliberal body politic. I trace a similar development here. My work also seeks to capture the player’s movements via autoethnographic writing that communicates the viscerally and intensity of the experience. The FPS is framed as capable of giving insight into both the present and the future of our technological and political milieu and ‘sensorium,’ in Walter Benjamin’s terms. In its valorisation of the individual and production of insecurity to incite action, this project argues that the FPS is a symbolic form of immunitary neoliberal governmentality