MAVIS: Multi-Stem Audio Visualisation in Immersive Spaces Framework

Abstract

Abstract The visualisation of music has gained traction in both research and musical composition in recent years. The increased accessibility to immersive technologies, such as virtual reality (VR) and other forms of mixed reality (MR), lend themselves to the examination of how visualisation can impact the perception of audio virtual worlds. In this paper, we propose the MAVIS (Multi-stem Audio Visualisation in Immersive Spaces) design framework, an approach to generating a visualisation of multi-stem structured orchestral music in a virtual world. This research explores the impact on participants’ interaction with an orchestral musical composition through the use of a two framework iterations informed by use cases. The resulting final design structure outlined in this article points towards constructing multi-stem virtual orchestral experiences through three pillars: semantic consistency, spatial agency, and complexity control. Whilst this research serves to propose a design intervention, future work requires a more extensive participant testing approach, coupled with an exploration of additional multimodal analysis. Keywords: immersive visualisation; virtual reality; multi-stem audi