Soma: live performance where congruent musical, visual, and proprioceptive stimuli fuse to form a combined aesthetic narrative

Artists and scientists have long had an interest in the relationship between music and visual art. Today, many occupy themselves with correlated animation and music, called 'visual music'. Established tools and paradigms for performing live visual music however, have several limitations: Virtually no user interface exists, with an expressivity comparable to live musical performance. Mappings between music and visuals are typically reduced to the music‘s beat and amplitude being statically associated to the visuals, disallowing close audiovisual congruence, tension and release, and suspended expectation in narratives. Collaborative performance, common in other live art, is mostly absent due to technical limitations. Preparing or improvising performances is complicated, often requiring software development. This thesis addresses these, through a transdisciplinary integration of findings from several research areas, detailing the resulting ideas, and their implementation in a novel system: Musical instruments are used as the primary control data source, accurately encoding all musical gestures of each performer. The advanced embodied knowledge musicians have of their instruments, allows increased expressivity, the full control data bandwidth allows high mapping complexity, while musicians‘ collaborative performance familiarity may translate to visual music performance. The conduct of Mutable Mapping, gradually creating, destroying and altering mappings, may allow for a narrative in mapping during performance. The art form of Soma, in which correlated auditory, visual and proprioceptive stimulus form a combined narrative, builds on knowledge that performers and audiences are more engaged in performance requiring advanced motor knowledge, and when congruent percepts across modalities coincide. Preparing and improvising is simplified, through re-adapting the Processing programming language for artists to behave as a plug-in API, thus encapsulating complexity in modules, which may be dynamically layered during performance. Design research methodology is employed during development and evaluation, while introducing the additional viewpoint of ethnography during evaluation, engaging musicians, audience and visuals performers