Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction.

This thesis contributes to the field of embodied and multimodal interaction by presenting the development of different original interactive systems. Using a constructive approach, a variety of real-time user interaction situations were designed and tested, two cases of human-virtual character bodily interaction, two interactive sonifications of trampoline jumping, collaborative interaction in mobile music performance and tangible and tactile interaction with virtual sounds. While diverse in terms of application, all the explored interaction techniques belong to the context of augmentation and are grounded in the theory of embodiment and strategies for natural human-computer interaction (HCI). The cases have been contextualized within the umbrella of enaction, a paradigm of cognitive science that addresses the user as an embodied agent situated in an environment and coupled to it through sensorimotor activity. This activity of sensing and action is studied through different modalities: auditory, tactile and visual and combinations of these. The designed applications aim at a natural interaction with the system, being full-body, tangible and spatially aware. Particularly sonic interaction has been explored in the context of music creation, sports and auditory display. These technology-mediated scenarios are evaluated in order to understand what the adopted interaction techniques can bring to the user experience, how they modify impressions and enjoyment. The publications also discuss the enabling technologies used for the development, including motion tracking and programmed hardware for the tactile-sonic interaction and sonic and tangible interaction. Results show that combining full-body interaction with auditory augmentation and sonic interaction can modify the perception, observed behavior and emotion during the experience. Using spatial interaction together with tangible interaction or tactile feedback provides for a multimodal experience of exploring a mixed reality environment where audio can be accessed and manipulated with natural interaction. Embodied and spatial interaction brings playfulness to a mobile music improvisation, shifting the focus of the experience from music-making towards movement-based gaming. Finally, two novel implementations of full-body interaction based on the enactive paradigm are presented. In these designed scenarios of enaction the participant is motion tracked and a virtual character rendered as a stick figure is displayed in front of her on a screen. Results from the user studies show how the involvement of the body is crucial in understanding the behavior of a virtual character or a digital representation of the self in a gaming scenario

The relationship between individual differences and human territoriality, within a simulated environment

Territoriality is defined as the acquisition of land or resources, such as food or mates, and securing these resources from actual or perceived threat. Many animals show differences in the expression of territoriality based on sex and personality differences, this study examined the extent these factors mediate territoriality usage, and what form it expresses itself in human participants. First gender (as identified by self-report questionnaire) and personality (scores attained from the Big Five Inventory Questionnaire), were examined, and how this altered in game territorial behaviours. It was found that females tended to use more non-aggressive territorial behaviours than males, and displayed a greater number of overall territorial behaviours than males. These results are consistent with an evolutionary theory of territoriality, whereby females tend towards more passive territorial behaviours, as opposed to males who tend towards more aggressive territoriality. Surprisingly, males did not show this inverse relation of increased aggressive territorial behaviours within this simulation.Secondly, power asymmetries were examined between avatars, based on predictions by Game Theory. It was found that participants assigned to smaller avatars displayed significantly more withdrawing behaviours in general, and in particular towards larger avatars, as compared to the individuals controlling larger avatars. Furthermore smaller avatars used more passive territorial behaviours than larger avatars. This is consistent with a Game Theory approach to territoriality.Finally winner and loser effects were examined in how they mediate emotional word usage in a short storytelling activity. It was found that low survival time is the best predictor of negative emotional word usage, and as survival time increase, negative emotional word usage decreased

In Gameplay : the invariant structures and varieties of the video game gameplay experience

This dissertation is a multidisciplinary study on video game gameplay as an autonomous form of vernacular experience. Plays and games are traditional research subjects in folkloristics, but commercial video games have not been studied yet. For this reason, methods and concepts of the folkloristic research tradition have remained unknown in contemporary games studies. This thesis combines folkloristics, game studies and phenomenological enactive cognitive science in its investigations into player–game interaction and the video game gameplay experience at large. In this dissertation, three representative survey samples (N=2,594, N=845, N=1,053) on “Rewarding gameplay experience” are analyzed using statistical analysis methods. The samples were collected in 2014–2017 from Finnish and Danish adult populations. This dissertation also analyzes data from 32 interviews, through which the survey respondents’ gameplay preferences, gaming memories, and motivations to play were further investigated. By combining statistical and qualitative data analyses, this work puts forward a mixed-methods research strategy and discusses how the findings relate to prior game research from several disciplines and schools of thought. Based on theoretical discussions, this dissertation argues that the video game gameplay experience as a cultural phenomenon consists of eight invariants in relation to which each individual gameplay experience can be interpreted: The player must demonstrate a lusory attitude (i), and a motivation to play (ii). The gameplay experience consists of explorative and coordinative practices (iii), which engender a change in the player’s self-experience (iv). This change renders the gameplay experience inherently emotional (v) and performative (vi) in relation to the gameworld (vii). The gameplay experience has the dramatic structure of a prototypical narrative (viii) although a game as an object cannot be regarded a narrative in itself. As a key result of factor analytical studies and qualitative interview analyses, a novel approach to understanding player–game interaction is put forward. An original gameplay preference research tool and a player typology are introduced. This work argues, that, although video games as commercial products would not be intuitive research subjects for folkloristics, video game gameplay, player–game interaction, and the traditions in experiencing and narrating gameplay do not differ drastically from those of traditional social games. In contrast to this, all forms of gameplay are argued to be manifestations of the same vernacular phenomenon. Indeed, folkloristic research could pay more attention to how culture is experienced, modified, varied and expressed, regardless of whether the research subject is a commercial product or not.Käsillä oleva väitöskirja on monitieteellinen tutkimus videopelien pelaamisesta itsenäisenä kansanomaisen kokemuksen muotona. Pelien ja leikkien tutkimus on perinteikäs tutkimusaihe folkloristiikassa, mutta kaupallisten videopelien tutkimusta ei ole juuri tehty. Tästä syystä folkloristiikan tutkimusmenetelmät ja -käsitteet ovat jääneet tuntemattomaksi nykyaikaisessa pelitutkimuksessa. Tutkimus yhdistää folkloristiikan ja pelitutkimuksen näkökulmien lisäksi enaktiivisen kognition fenomenologista teoriaa pelaaja–peli-vuorovaikutuksen tutkimukseen sekä pelikokemuksen analyysiin. Tutkimuksessa analysoidaan tilastotieteellisin menetelmin kolmea aikuisväestöä edustavaa ”Palkitseva pelikokemus” -kyselytutkimusaineistoa (N=2,594, N=845, N=1,053), jotka kerättiin Suomesta ja Tanskasta vuosina 2014–2017. Kyselytutkimusaineiston rinnalla analysoidaan 32 teemahaastattelun aineistoa. Haastatteluilla tuotettiin syvempää ymmärrystä kyselyyn vastanneiden henkilöiden pelimieltymyksistä, pelimuistoista ja pelimotivaatioista. Tilastoaineiston ja haastatteluaineiston analyysi tuodaan yhteen monimenetelmällisellä ja dialogisella tutkimusotteella, joka yhdistää havainnot usealla eri tutkimusalalla tehtyyn pelitutkimukseen. Teoreettisen analyysin tuloksena argumentoidaan, että videopelien pelikokemusta ilmiönä määrittää kahdeksan muuttumatonta ominaisuutta, joiden suhteen kunkin yksittäisen pelikokemuksen ainutlaatuisuutta voidaan tarkastella: Pelaajalla tulee olla leikkisä asenne (i) ja motivaatio pelaamiseen (ii). Pelaamisen kokemus rakentuu tutkivista ja suorittavista käyntänteistä (iii), jotka tuovat väliaikaisen muutoksen pelaavan henkilön minäkokemukseen (iv). Tämän muutoksen myötä pelaajuudesta muodostuu emotionaalinen (v) ja performatiivinen (vi) positio suhteessa pelimaailmaan (vii). Näin syntyvän omakohtaisen pelikokemuksen rakenne vastaa kertomuksen dramaattista perusrakennetta (viii), vaikka peliä itsessään ei voida pitää kertomuksena. Tutkimuksen empiirisenä tuloksena esitellään faktorianalyyttisiin tapaustutkimuksiin ja laadullisten aineistojen analyysiin perustuva uudenlainen näkökulma ja menetelmä pelaaja–peli-vuorovaikutuksen ja pelimieltymyksen tutkimukseen, sekä edelliseen perustuva pelaajatyyppiluokittelu. Samalla väitetään, että vaikka videopelit kaupallisina esineinä eivät olisi folkloristiikan tutkimuskohteita, videopelien pelaaminen, pelaaja–peli-vuorovaikutus ja pelien kokemisen tavat eivät eroa ratkaisevasti pihaleikeistä vaan ovat saman kansanomaisen ilmiön esiintymiä. Folkloristisen tutkimuksen soisikin kiinnittävän nykyistä painokkaampaa huomiota kulttuurin kokemisen, muokkaamisen ja ilmaisun tapoihin riippumatta siitä, onko tarkastelun kohteena kaupallinen tuote vai ei

The Translocal Event and the Polyrhythmic Diagram

This thesis identifies and analyses the key creative protocols in translocal performance practice, and ends with suggestions for new forms of transversal live and mediated performance practice, informed by theory. It argues that ontologies of emergence in dynamic systems nourish contemporary practice in the digital arts. Feedback in self-organised, recursive systems and organisms elicit change, and change transforms. The arguments trace concepts from chaos and complexity theory to virtual multiplicity, relationality, intuition and individuation (in the work of Bergson, Deleuze, Guattari, Simondon, Massumi, and other process theorists). It then examines the intersection of methodologies in philosophy, science and art and the radical contingencies implicit in the technicity of real-time, collaborative composition. Simultaneous forces or tendencies such as perception/memory, content/ expression and instinct/intellect produce composites (experience, meaning, and intuition- respectively) that affect the sensation of interplay. The translocal event is itself a diagram - an interstice between the forces of the local and the global, between the tendencies of the individual and the collective. The translocal is a point of reference for exploring the distribution of affect, parameters of control and emergent aesthetics. Translocal interplay, enabled by digital technologies and network protocols, is ontogenetic and autopoietic; diagrammatic and synaesthetic; intuitive and transductive. KeyWorx is a software application developed for realtime, distributed, multimodal media processing. As a technological tool created by artists, KeyWorx supports this intuitive type of creative experience: a real-time, translocal “jamming” that transduces the lived experience of a “biogram,” a synaesthetic hinge-dimension. The emerging aesthetics are processual – intuitive, diagrammatic and transversal

Fictional Practices of Spirituality I: Interactive Media

"Fictional Practices of Spirituality" provides critical insight into the implementation of belief, mysticism, religion, and spirituality into worlds of fiction, be it interactive or non-interactive. This first volume focuses on interactive, virtual worlds - may that be the digital realms of video games and VR applications or the imaginary spaces of life action role-playing and soul-searching practices. It features analyses of spirituality as gameplay facilitator, sacred spaces and architecture in video game geography, religion in video games and spiritual acts and their dramaturgic function in video games, tabletop, or LARP, among other topics. The contributors offer a first-time ever comprehensive overview of play-rites as spiritual incentives and playful spirituality in various medial incarnations

Socially Responsible Games

This paper analyzes the ethics of producing video games with violent, militaristic, or stereotypical themes. The authors argue that video games shape the social ideologies that players hold and that the current video game industry promotes unjust ideas. The authors attempted to design a socially responsible game and analyze its ethical content. Social responsibility in video games is a difficult task due to market pressures, technological limitations, and a lack of understanding of the issue within the video game industry

Sonic interactions in virtual environments

This book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Examining participatory sense-making frames: how autonomous patterns of being together emerge in recurrent social interaction

This thesis investigates how recurrent face-to-face social interactions engender relatively invariant patterns of being together that cause those who instantiate them to act in ways that support their reproduction. Existing accounts within both cognitive science and sociology offer important insights into the consideration of patterns of being together. However, given their explanatory strategies, they struggle to integrate both ‘social’ and ‘individual’ levels of explanation. Herein a compatibilist account is developed, intended as a ‘third way’ that obviates the limitations of existing accounts. This compatibilist account — by integrating insights from across disciplines and theoretical frameworks — develops a philosophical vocabulary with which to maintain explanatory consistency when articulating patterns of being together and moving between individual and social levels of explanation. It relies heavily on an extension of the enactive notion of autonomous habits to the social domain, redescribing patterns of being together as habituated participatory sense-making frames. Participatory sense-making frames result from processes of coenhabiting, i.e. processes of ongoing social habit making implicit in the dynamics of recurrent social interactions. Such processes are one primary means by which we produce and reproduce the relatively stable forms that characterise our shared worlds. These habituated frames embed much of the normativity of social life, and can serve our felicitous coordinations therein, allowing us to feel well situated, particularly in contexts within which we have some history of interacting. However, when they are not well aligned, they lead to tensions that result in either the production of novel frames or breakdowns in social interaction. The account developed has implications for many domains of human action, from psychotherapy to epistemology, and from critical studies to the development of political and ecological praxes

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments