Mixed Realities: a live collaborative musical performance

In the presented work, a live-rendered percussionist is transformed into a virtual game character and performs a piece along digital avatars created from recordings (audio and motion-capture) of other members of an ensemble, while audience members can observe the collaborative performance through VR headsets. To create a cohesive and compelling result, the auditory expectations of the listeners need to be considered in terms of acoustic integrity between real and virtual sources and spatial impression of each avatar performer. This paper presents an overview of the workflow and motivations behind this pilot experiment of a novel musical experience, laying down the foundations for future subjective studies into collaborative music performances using virtual and augmented reality headsets. Particular focus is given to the technical challenges concerning the audio material, the perspectives of each participant role, and the qualitative impressions of musicians and audience

Audio for Virtual, Augmented and Mixed Realities: Proceedings of ICSA 2019 ; 5th International Conference on Spatial Audio ; September 26th to 28th, 2019, Ilmenau, Germany

The ICSA 2019 focuses on a multidisciplinary bringing together of developers, scientists, users, and content creators of and for spatial audio systems and services. A special focus is on audio for so-called virtual, augmented, and mixed realities. The fields of ICSA 2019 are: - Development and scientific investigation of technical systems and services for spatial audio recording, processing and reproduction / - Creation of content for reproduction via spatial audio systems and services / - Use and application of spatial audio systems and content presentation services / - Media impact of content and spatial audio systems and services from the point of view of media science. The ICSA 2019 is organized by VDT and TU Ilmenau with support of Fraunhofer Institute for Digital Media Technology IDMT