PhDWhile physics-based synthesis offers a wide range of benefits in the real-time generation
of sound for interactive environments, it is difficult to incorporate nuanced
and complex behaviour that enhances the sound in a narrative or aesthetic context.
The work presented in this thesis explores real-time human performance as a means
of stylistically augmenting computational sound models. Transdisciplinary in nature,
this thesis builds upon previous work in sound synthesis, film sound theory and physical
sound interaction. Two levels on which human performance can enhance the
aesthetic value of computational models are investigated: first, in the real-time manipulation
of an idiosyncratic parameter space to generate unique sound effects, and
second, in the performance of physical source models in synchrony with moving images.
In the former, various mapping techniques were evaluated to control a model of
a creaking door based on a proposed extension of practical synthesis techniques. In
the latter, audio post-production professionals with extensive experience in performing
Foley were asked to perform the soundtrack to a physics-based animation using
bespoke physical interfaces and synthesis engines. The generated dataset was used to
gain insights into stylistic features afforded by performed sound synchronisation, and
potential ways of integrating them into an interactive environment such as a game
engine.
Interacting with practical synthesis models that have extended to incorporate performability
enables rapid generation of unique and expressive sound effects, while
maintaining a believable source-sound relationship. Performatively authoring behaviours
of sound models makes it possible to enhance the relationship between sound
and image (both stylistically and perceptually) in ways precluded by one-to-one mappings
between physics-based parameters. Mediation layers are required in order to
facilitate performed behaviour: in the design of the model on one hand, and in the
integration of such behaviours into interactive environments on the other. This thesis
provides some examples of how such a system could be implemented. Furthermore,
some interesting observations are made regarding the design of physical interfaces for
performing environmental sound, and the creative exploitation of model constraints.Engineering and Physical Sciences Research Council
(EPSRC) Doctoral Training Centre in Media and Arts Technology (ref: EP/G03723X/1)
