Time, Space, Memory: A portfolio of acousmatic compositions

This portfolio comprises of a collection of acousmatic works which investigate the role of source bonding in music – the tendency of listeners to relate sounds to their real-world sources and the signifying implication of such a link – with a particular focus on how spatial design can contribute towards source-bonding in the music’s perception as a holistic spatio-sonic entity. A number of compositional strategies, multichannel formats and spatial audio technologies are investigated, with their merits assessed based on their suitability for shaping the qualities of musical space explored. The discussion in this commentary will show how these holistic spaces can have similar qualities of perceived ‘reality’ and ‘abstraction’ to the individual sounds, and how this is investigated in the musical works. I shall also show how the contrasting environmental qualities of these spaces became a source of of inspiration for structuring the development of my music, and how they might evoke subsequent meaning in their experience based on the listener’s understanding of the spatial source bonds

Sonic Choreosophia: a cross-disciplinary investigation on sound and movement practices

This thesis is the account of cross-disciplinary research that explores spatial audio experiences in multimodal contexts. The practice of arranging dynamical modifications of spatial attributes of sound to create impressions of movement through sound has been applied to dance choreography and theatre. Using wave field synthesis and ambisonics technologies for spatial audio sound playback, two projects have been created: Stranded (2013), a joint choreography for three dancers and sonic movement in collaboration with choreographer Jalianne Li, and I Hear You See Me (2014), an audiovisual installation featuring participatory theatre, sonic movement, and motion graphics, in collaboration with theatre artist Silvia Mercuriali and visual artist Simon Wilkinson. These works are the outcome of a complex collaborative exchange between the author and the mentioned artists and a comparison at multiple levels (aesthetic, technical, cultural) between the different disciplines involved, and propose alternative reflections about spatial audio composition. For example, the choreographic ideas of Li, the aesthetics and movement studies of Rudolf Laban, the works and writing of choreographers Mary Wigman, Merce Cunningham and Pina Bausch have all been used to evaluate the kinetic power of sonic movement and its strengths measured against the clarity and immediacy of a dancing body. The participatory strategies of Mercuriali’s theatre, the composite works by Len Lye's, Oskar Fischinger's audiovisual experiments, and historical and contemporary examples from kinetic and installation art have all helped to bring forward a further reflection over a shift of function of sound, from essence of a composition to instrument for realising a kinetic idea. Highlighting the necessity of a multimodal context when using spatial audio, but limiting the idea of a Sonic Choreosophia to a simple suggestion, this thesis thus documents a novel approach of using sound to create movement per se, and its potential for further development

Quais as potencialidades da criação de um videoclip em Realidade Virtual?

The focus of this dissertation is in the potentiality of the creation of a music video in Virtual Reality (VR). As such, it was analyzed in the theoretical framework what are music videos, what is VR and, more specifically, what are music videos in VR, which is a unresearched topic in the academic field, to which this dissertation provides some basis for feature research in this area. A music video for VR was made for this dissertation as to better understand the potentialities of the medium. The music video was made under the conventional phases of audiovisual production. Then user tests were made for this music video in order to evaluate its characteristics. The tests were made with 3 independent samples because the music video was shown in desktop 360-2D, mobile-360-2D, and VR on a Head Mounted Display (HMD). This was done as to better understand the potentialities of VR by comparing it to other mediums. Each participant had to answer to a pre-test questionnaire, and to a post-test questionnaire. The analysis of the results showed that although VR offers the added benefit of immersion it also poses its own challenges and a completely new approach to music video making.O foco desta dissertação é na potencialidade de criação de videoclips em Realidade Virtual. Como tal, foi analisado no enquadramento teórico o que são videoclips, o que é a Realidade Virtual e, mais especificamente, o que são videoclips em Realidade Virtual, tópico sobre o qual não existe muita investigação no meio académico, de maneira a que esta dissertação serve como base para investigações futuras. Um videoclip foi feito para Realidade Virtual para a investigação da qual esta dissertação faz parte, a fim de perceber as potencialidades do meio. O videoclip foi criado com base nas fases convencionais de produção audiovisual. Feito isto foram feitos testes de utilizador para avaliar as características do videoclip. Estes testes foram feitos em 3 amostras independentes cada uma para um formato diferente de visualização de videoclips em Realidade Virtual, desktop-360-2D, mobile-360-2D e em Realidade Virtual com um Head Mounted Display. Isto foi feito de modo a perceber melhor as potencialidades da Realidade Virtual comparativamente com os outros meios. Cada participante teve de responder a um questionário pré-teste e a um questionário pós-teste. A análise dos resultados mostrou que apesar da Realidade Virtual oferecer o benefício da imersão, ela tem os seus desafios, e requere uma abordagem completamente nova para a realização de videoclips.Mestrado em Comunicação Multimédi

Dislocations in sound design for 3-d films: sound design and the 3-d cinematic experience

Since the success of James Cameron’s Avatar (2009),1 the feature film industry has embraced 3-D feature film technology. With 3-D films now setting a new benchmark for contemporary cinemagoers, the primary focus is directed towards these new stunning visuals. Sound is often neglected until the final filmmaking process as the visuals are taking up much of the film budget. 3-D has changed the relationship between the imagery and the accompanying soundtrack, losing aspects of the cohesive union compared with 2-D film. Having designed sound effects on Australia’s first digital animated 3-D film, Legend of the Guardians: The Owls of Ga’Hoole (2010),2 and several internationally released 3-D films since, it became apparent to me that the visuals are evolving technologically and artistically at a rate far greater than the soundtrack. This is creating a dislocation between the image and the soundtrack. Although cinema sound technology companies are trialing and releasing new ‘immersive’ technologies, they are not necessarily addressing the spatial relationship between the images and soundtracks of 3-D digital films. Through first hand experience, I question many of the working methodologies currently employed within the production and creation of the soundtrack for 3-D films. There is limited documentation on sound design within the 3-D feature film context, and as such, there are no rules or standards associated with this new practice. Sound designers and film sound mixers are continuing to use previous 2-D work practices in cinema sound, with limited and cautious experimentation of spatial sound design for 3-D. Although emerging technologies are capable of providing a superior and ‘more immersive’ soundtrack than previous formats, this does not necessarily mean that they provide an ideal solution for 3-D film. Indeed the film industry and cinema managers are showing some resistance in adopting these technologies, despite the push from technology vendors. Through practice-led research, I propose to research and question the following:Does the contemporary soundtrack suit 3-D films? ; Has sound technology used in 2-D film changed with the introduction of 3-D film? If it has, is this technology an ideal solution, or are further technical developments needed to allow greater creativity and cohesiveness of 3-D film sound design? ; How might industry practices need to develop in order to accommodate the increased dimension and image depth of 3-D visuals? ; Does a language exist to describe spatial sound design in 3-D cinema? ; What is the audience awareness of emerging film technologies? And what does this mean for filmmakers and the cinema? ; Looking beyond contemporary cinema practices, is there an alternative approach to creating a soundtrack that better represents the accompanying 3-D imagery

Future spatial audio : Subjective evaluation of 3D surround systems

Current surround systems are being developed to include height channels to provide the listener with a 3D listening experience. It is not well understood the impact the height channels will have on the listening experience and aspects associated with multichannel reproduction like localisation and envelopment or if there are any new subjective attributes concerned with 3D surround systems. Therefore in this research subjective factors like localisation and envelopment were investigated and then descriptive analysis was used. In terms of localisation it was found that for sources panned in the median plane localisation accuracy was not improved with higher order ambisonics. However for sources in the frontal plane higher order ambisonics improves localisation accuracy for elevated sound sources. It was also found that for a simulation of a number of 2D and 3D surround systems, using a decorrelated noise signal to simulate a diffuse soundfield, there was no improvement in envelopment with the addition of height. On the other hand height was found to improve the perception of envelopment with the use of 3D recorded sound scenes, although for an applause sample which had similar properties to that of the decorrelated noise sample there was no significant difference between 2D and 3D systems. Five attribute scales emerged from the descriptive analysis of which it was found that there were significant differences between 2D and 3D systems using the attribute scale size for both ambisonics and VBAP rendered systems. Also 3D higher order ambisonics significantly enhances the perception of presence. A final principal component analysis found that there were 2 factors which characterised the ambisonic rendered systems and 3 factors which characterised the VBAP rendered sound scenes. This suggests that the derived scales need to be used with a wider number of sound scenes in order to fully validate them

Multimodal Perception of Auditoria: Influence of Auditory and Visual Factors on Preference

The enjoyment of a music performance is a multisensory experience, of which auditory and visual senses play the most important parts in conveying the content of the concerts. This thesis investigates the effects of and relationships between various auditory and visual factors on subjective preference, with an emphasis on the rarely-studied visual preference. The thesis includes four subjective evaluation experiments (all using head-mounted virtual reality display and headphones audio playback, 30 to 33 volunteers each) and one online survey (153 responses). The experimental method of virtual reality display and digital audio playback allows each factor to be individually controlled and tested, which was never possible with traditional methods, but still provides a reasonable sense of space and realism. Auditory factors considered in the thesis include sound pressure level and reverberation time, while visual factors include interior design colour, distance from the stage, lateral angle from the concert hall mid-plane, vertical angle from stage level, and visual obstruction. The effects of factors were studied using orthogonal control, and verified with realistic models and alternative methods with larger sample. Results include a prediction model that accounts for the effects and relationships of all investigated factors, and a practical tool for design/evaluation of auditorium seating layout

XR, music and neurodiversity: design and application of new mixed reality technologies that facilitate musical intervention for children with autism spectrum conditions

This thesis, accompanied by the practice outputs,investigates sensory integration, social interaction and creativity through a newly developed VR-musical interface designed exclusively for children with a high-functioning autism spectrum condition (ASC).The results aim to contribute to the limited expanse of literature and research surrounding Virtual Reality (VR) musical interventions and Immersive Virtual Environments (IVEs) designed to support individuals with neurodevelopmental conditions. The author has developed bespoke hardware, software and a new methodology to conduct field investigations. These outputs include a Virtual Immersive Musical Reality Intervention (ViMRI) protocol, a Supplemental Personalised, immersive Musical Experience(SPiME) programme, the Assisted Real-time Three-dimensional Immersive Musical Intervention System’ (ARTIMIS) and a bespoke (and fully configurable) ‘Creative immersive interactive Musical Software’ application (CiiMS). The outputs are each implemented within a series of institutional investigations of 18 autistic child participants. Four groups are evaluated using newly developed virtual assessment and scoring mechanisms devised exclusively from long-established rating scales. Key quantitative indicators from the datasets demonstrate consistent findings and significant improvements for individual preferences (likes), fear reduction efficacy, and social interaction. Six individual case studies present positive qualitative results demonstrating improved decision-making and sensorimotor processing. The preliminary research trials further indicate that using this virtual-reality music technology system and newly developed protocols produces notable improvements for participants with an ASC. More significantly, there is evidence that the supplemental technology facilitates a reduction in psychological anxiety and improvements in dexterity. The virtual music composition and improvisation system presented here require further extensive testing in different spheres for proof of concept

The generative, analytic and instructional capacities of sound in architecture : fundamentals, tools and evaluation of a design methodology

The disciplines of space and time form two domains to which it is daring to compare, since it is obvious that they are of a different nature. Music happens in time, while architecture happens in space. However, from the first treatises on both architecture and music, repeated calls for comparison, complementarity and influence of both disciplines can be read, at least to the observation of certain common orders between the two domains. In this doctoral thesis we do not question this whole theoretical corpus that has been enriching the relationship between both disciplines. We received it and joined that stream of knowledge. What we do notice, however, is the almost impertinent question that follows: can sound help the architect in his daily tasks? And, therefore, what are the contributions of sound to the architect? To do this we must seek the connection in the principles of both arts, where we can detach ourselves from time and space, and approach the most universal of art forms. The architect, in his daily work, is faced with three particular tasks: the architectural project, the architectural analysis and the teaching of architecture. Each of the three tasks is connected with the other two tasks: the project is carried out again with the analysis and transmitted to the new architect; the analysis supports the project decisions and gives tools to the disciple; and the teaching has the project as its purpose and the analysis as its method. The thesis presented here shows what sound offers to the task of the project, to that of analysis and to that of teaching. These three tasks are approached from three premises: theoretical foundations, tools and evaluation. The interaction of the three tasks with the three premises gives rise to nine lines of work that articulate the chapters of the thesis. The first, fourth and seventh chapters approach the three tasks from the premise of theoretical foundations, foundations that perhaps because they are obvious, have been ignored or overlooked but which constitute the nature of both disciplines. The first shows, by the hand of two 20th century authors - the architect Dom Hans van der Laan and the composer Olivier Messiaen - that creation in both disciplines is of a systematic nature. The fourth one revaluates the analytical systems of representation of form both in architecture and in music which, starting with the basic characteristics of its elements, lead to a symbolic notation and a tool for the analysis of the work: the plan and the score. The seventh introduces the student of architecture to the growing separation between music and architecture that has been accentuated to this day. The second, fifth and eighth chapters approach the three particular tasks from the premise of tools, working tools that help to understand more directly the influence of architecture on sound. The second places virtual reality and auralization techniques at the service of the architectural and urban planning project, enhancing the sound experience in these projects. The fifth deals with the acoustic analysis of exterior spaces and their relationship with the urban configuration of these spaces. The eighth section presents the study of acoustic heritage as an educational tool. The third, sixth and ninth chapters deal with the three tasks from the premise of evaluation, a check that ensures the influence of sound on them through teaching experiments. The third argues and exemplifies that a sound landscape can be the engine and generator of an architectural design. The sixth one reviews the methods for evaluating the subjective and objective parameters of architectural acoustics. The ninth shows that in teaching sound to architects, "learning by listening" should be given priority over "passive learning".Las disciplinas del espacio y del tiempo forman dos dominios a los que resulta atrevido comparar, pues es obvio que son de naturaleza distinta. La música ocurre en el tiempo, mientras que la arquitectura en el espacio. No obstante, desde los primeros tratados tanto de arquitectura como de música, se pueden leer repetidas llamadas a la comparación, al complemento y a la influencia de ambas disciplinas, cuanto menos a la constatación de ciertos órdenes comunes entre ambos dominios. En esta tesis doctoral no ponemos en cuestión todo este corpus teórico que ha venido enriqueciendo la relación entre ambas disciplinas. La recibimos y nos unimos a esa corriente de conocimiento. En lo que sí reparamos, en cambio, es en la pregunta casi impertinente que surge seguidamente: ¿puede el sonido ayudar al arquitecto en sus tareas diarias? Y, por tanto, ¿cuáles son las contribuciones del sonido para el arquitecto? Para ello debemos buscar la conexión en los principios de ambas artes, allí donde podemos despegarnos del tiempo y del espacio, y acercarnos a la más universal de las formas de arte. El arquitecto, en su tarea diaria, se enfrenta a tres tareas particulares: el proyecto arquitectónico, el análisis arquitectónico y la enseñanza de la arquitectura. Cada una de las tres tareas está conectada con las otras dos: el proyecto se reconduce con el análisis y se transmite al nuevo arquitecto; el análisis soporta las decisiones de proyecto y da herramientas al discípulo; y la enseñanza tiene como fin el proyecto y como método el análisis. La tesis aquí presentada pone de manifiesto lo que el sonido ofrece a la tarea del proyecto, a la del análisis y a la de la enseñanza. Estas tres tareas son abordadas desde tres premisas: los fundamentos teóricos, las herramientas y la evaluación. La interacción de las tres tareas con las tres premisas da lugar a nueve líneas de trabajo que articulan los capítulos de la tesis. Los capítulos primero, cuarto y séptimo abordan las tres tareas desde la premisa de los fundamentos teóricos, fundamentos que quizá por ser obvios, se han obviado o pasado por alto pero que constituyen la naturaleza de ambas disciplinas. El primero muestra, de la mano de dos autores del siglo XX ?el arquitecto Dom Hans van der Laan y el compositor Olivier Messiaen- que la creación en ambas disciplinas es de naturaleza sistemática. El cuarto revaloriza los sistemas analíticos de representación de la forma tanto en arquitectura como en música que, empezando por las características básicas de sus elementos, conducen a una notación simbólica y una herramienta de análisis de la obra: el plano y la partitura. El séptimo presenta al estudiante de arquitectura la creciente separación entre la música y la arquitectura que se ha venido acentuando hasta nuestros días. Los capítulos segundo, quinto y octavo abordan las tres tareas particulares desde la premisa de las herramientas, útiles de trabajo que ayudan a comprender de modo más directo la influencia de la arquitectura en el sonido. El segundo sitúa la realidad virtual y las técnicas de auralización al servicio del proyecto de arquitectura y urbanismo, potenciando la experiencia sonora en estos proyectos. El quinto aborda el análisis acústico de espacios exteriores y su relación con la configuración urbana de estos espacios. El octavo presenta el estudio del patrimonio acústico como herramienta pedagógica. Los capítulos tercero, sexto y noveno abordan las tres tareas desde la premisa de la evaluación, comprobación que asegura mediante experimentos docentes la influencia del sonido en ellas. El tercero argumenta y ejemplifica que un paisaje sonoro puede ser el motor y generador de un diseño arquitectónico. El sexto realiza una revisión de los métodos de evaluación de los parámetros subjetivos y objetivos de la acústica arquitectónica. El noveno muestra que en la enseñanza del sonido para los arquitectos debe priorizarse "aprender escuchando" antes que el "aprendizaje pasivo"