How Sound Effects Affect the Player in Video Games

The research paper talks about how sound effects affect a player in a video game. There are three different concepts that are important to how sound effects affect a player: synchresis, kinesonic synchresis, and kinesonic congruity. Synchresis is the fusion of sound and image which can lead to new or altered meanings to the original. Kinesonic synchresis is where sounds are fused with action instead of an image. Finally, kinesonic congruity is where there is congruence or a relation to the player’s action and the sound generated. Throughout the paper, there are examples of many video games that follow these concepts. The examples used to prove that these concepts are true are things that players go through in a video game. For the second half of the paper, the author talks about their sound design choices for their video game

Virtual Heritage: Audio design for immersive virtual environments using researched spatializers.

This thesis work is based on a Virtual Heritage project being developed by the Systems of Representation research group. The objective of the project is to create a showcase demonstration on how the virtual reality (VR) could be used as an application for tourism in the heritage sector. In this context, my task was to develop a concept and prototype of how 'spatialized' sound could be used in a VR application. The initial location chosen for the concept was the ancient heritage burial site of Sammallahdenmäki, one of the Finnish heritage sites listed in the UNESCO register of World Heritage Sites. The thesis, that is written from an audio designer's perspective, focuses on three aspects of this project. First is the sound design for the Virtual Heritage project and the second is the quality of currently available 'spatializer' plug-ins used for headphone listening. In order to evaluate the process of designing 3D audio for virtual environments, the methods and principles within binaural rendering, sound design and immersion must be understood. Therefore, functions and theories within audio spatialization and 3D audio design are reviewed. Audio designers working on virtual reality content need the best possible solutions for creating believable 3D audio experiences. However, while working on the Virtual Heritage project, we did not find any comparative studies made about commercially available spatializer plug-ins for Unity. Thus, it was unknown what plug-in would have been the best possible solution for 3D audio spatialization. Consequently, two tests were conducted during this thesis work. First was an online test measuring which spatializer would be the most highly rated, in terms of perceived directional precision when utilizing head-related transfer functions without reverb or room simulations. The second was a comparative test studying if a spatialized audio rendering would increase immersion compared to non-spatialized audio rendering, when tested with the Virtual Heritage demonstration. The central aim in the showcase demonstration was to create an immersive virtual environment where users would feel as if they were travelling from the present, back to the Bronze Age, in order to understand and learn about the location’s unique history via auditory storytelling. The project was implemented utilising the Unity game engine. The research on music and other sound content used in the project’s sonic environment is explained. Finally, results of the project work are discussed.Tämä opinnäytetyö perustuu Virtual Heritage projektityöhön, joka on tehty Systems of Representation tutkimusryhmälle. Projektin tavoite on luoda malliesimerkki siitä, miten virtuaalitodellisuutta voitaisiin käyttää hyväksi turismisovelluksissa. Esimerkkikohteeksi projektille oli valittu Sammallahdenmäen hautaröykkiöt, joka on hyväksytty mukaan UNESCON maailmanperintöluetteloon. Tehtäväni oli toteuttaa Unity pelimoottorilla prototyyppi, jossa kartoitetaan virtuaalisen tilaäänen käyttömahdollisuuksia kyseisen teeman ympärillä. Opinnäyte on kirjoitettu äänisuunnittelijan näkökulmasta keskittyen kolmeen projektityöhön liittyvään keskeiseen osaan: prototyypin äänisuunnitteluun, immersion käsitteeseen sekä spatialisointi liitännäisten (plug-in) toimintaan ja laatuun. Virtuaalitodellisuuksiin sisältöä tuottavana äänisuunnittelijana tarvitsin parhaat mahdolliset työkalut uskottavan 3D äänimaailman luomiseen. Virtual Heritage projektia työstettäessä kävi kuitenkin ilmi, että ajankohtaista vertailevaa tutkimusta spatialisointi liitännäisten laatueroista ei ollut löydettävissä. Oli mahdotonta määritellä yksilöllisesti, mikä liitännäisistä on toimivin ratkaisu suurimmalle käyttäjäkunnalle. Täten oli tarpeellista toteuttaa kaksi tutkimusta. Ensimmäinen oli empiirinen verkkotutkimus, jolla arvioitiin spatialisointi liitännäisten suorituskykyä, kun mitataan subjektiivisesti äänen tilallisen sijoittumisen tarkkuutta kuulokekuuntelussa ilman kaikuprosessointeja. Tutkimuksessa parhaiten suoriutunut liitännäinen implementoitiin prototyyppiin. Toisessa kokeessa tutkittiin kuinka paljon implementoidun spatialisointi liitännäisen käyttäminen lisää virtuaalitodellisuuden immersiota verrattaessa spatialisoimattomaan ääneen, kun testialustana toimii Virtual Heritage prototyyppi. Projektin keskeisin tavoite oli luoda immersiivinen virtuaalitodellisuus, jossa käyttäjä voi kokea matkaavansa nykyajasta pronssikautiselle Sammallahdenmäelle ja oppia tällä tavoin kohteen ainutlaatuisesta historiasta äänikerronnan keinoin. Opinnäytetyössä esitellään äänikerronnan sisältöön ja toteutustapaan johtavat tutkimukset, tuotanto sekä ajatukset lopputuloksesta

Practical Design and Implementation of a CAVE Environment

CAVE systems are nowadays one of the best Virtual Reality (VR) immersive devices available for rendering virtual environments. Unfortunately, such kind of hardware is extremely expensive, complex and cum-bersome, thus limited in its spread. Several cheaper solutions already exist, but they implement usually only a subset of features of a professional CAVE (like a reduced number of wall-displays, no stereographic at all or with a low-quality support, low brightness, etc.). In this paper we describe how we have built a low cost CAVE with four screens (three walls and a floor), stereographic rendering and user tracking by only using hardware commonly available on the market and free software, we show the different solutions and work-around we implemented to solve the problems we encountered and we conclude with a fair evaluation of our system by using two applications we developed with it

Is Chalmers' Virtual Reality "Mirror Argument" Sound?

Extended reality devices provide users with unprecedented immersive and hybrid perceptual experiences, and users will act their bodies according to the information perceived. This shows that visual perception plays a crucial role in the formation and shaping of self-perception and spatial position. Users have a strong perceptual experience of their physical presence and self-perception in the real world as a result of their avatar perspective based on visual perception in a virtual hybrid environment, as is issued by Chalmers in his "Mirror Argument". However, users can still clearly differentiate their self and spatial experience between virtual and reality. To refute Chalmers' argument regarding perceptual experience in virtual reality, this article proposes a thought experiment called "Mary's Room in the Virtual Reality World" based on the experimental evidence of neuroscience. Finally, a possible future solution to this dilemma is presented

Incrementar la presencia en entornos virtuales en primera persona a través de interfaces auditivas: un acercamiento analítico al sonido y la música adaptativos

Tesis de la Universidad Complutense de Madrid, Facultad de Informática, leída el 25-11-2019The popularisation of virtual reality devices has brought with it an increased need of telepresence and player immersion in video games. This goals are often pursued through more realistic computer graphics and sound; however, invasive graphical user interfaces are still present in industry standard products for VR, even though previous research has advised against them in order to reach better results in immersion. Non-visual, multimodal communication channels are explored throughout this thesis as a means of reducing the amount of graphical elements needed in head-up displays while increasing telepresence. Thus, the main goals of this research are to find the optimal channels that allow for semantic communication without recurring to visual interfaces, while reducing the general number of extra-diegetic elements in a video game, and to develop a total of six software applications in order to validate the obtained knowledge in real-life scenarios. The central piece of software produced as a result of this process is called LitSens, and consists of an adaptive music generator which takes human emotions as inputs...La popularización de los dispositivos de realidad virtual ha traído consigo una mayor necesidad de presencia e inmersión para los jugadores de videojuegos. Habitualmentese intenta cumplir con dicha necesidad a través de gráficos y sonido por ordenador más realistas; no obstante, las interfaces gráficas de usuario muy invasivas aún son un estándar en la industria del videojuego de RV, incluso si se tiene en cuenta que varias investigaciones previas a la redacción de este texto recomiendan no utilizarlas para conseguir un resultado más inmersivo. A lo largo de esta tesis, varios canales de comunicación multimodales y no visuales son explorados con el fin de reducir la cantidad de elementos gráficos extradiegéticos necesarios en las capas de las interfaces gráficas de usuario destinadas a la representación de datos, todo ello mientras se logra un aumento de la sensación de presencia. Por tanto, los principales objetivos de esta investigación son encontrar los canales óptimos para efectuar comunicación semántica sin recurrir a interfaces visuales —a la vez que se reduce el número de elementos extradiegéticos en un videojuego— y desarrollar un total de seis aplicaciones con el objetivo de validar todo el conocimiento obtenido mediante prototipos similares a videojuegos comerciales. De todos ellos, el más importante es LitSens: un generador de música adaptativa que toma como entradas emociones humanas...Fac. de InformáticaTRUEunpu

"Knowing is Seeing:" The Digital Audio Workstation and the Visualization of Sound

The computers visual representation of sound has revolutionized the creation of music through the interface of the Digital Audio Workstation software (DAW). With the rise of DAW-based composition in popular music styles, many artists sole experience of musical creation is through the computer screen. I assert that the particular sonic visualizations of the DAW propagate certain assumptions about music, influencing aesthetics and adding new visually-based parameters to the creative process. I believe many of these new parameters are greatly indebted to the visual structures, interactional dictates and standardizations (such as the office metaphor depicted by operating systems such as Apples OS and Microsofts Windows) of the Graphical User Interface (GUI). Whether manipulating text, video or audio, a users interaction with the GUI is usually structured in the same mannerclicking on windows, icons and menus with a mouse-driven cursor. Focussing on the dialogs from the Reddit communities of Making hip-hop and EDM production, DAW user manuals, as well as interface design guidebooks, this dissertation will address the ways these visualizations and methods of working affect the workflow, composition style and musical conceptions of DAW-based producers