INSTRUMENT POSITION IN IMMERSIVE AUDIO: A STUDY ON GOOD PRACTICES AND COMPARISON WITH STEREO APPROACHES

O áudio espacial tem sido um foco na produção musical, impulsionando estratégias criativas na pós-produção. As tecnologias de som imersivo expandiram as possibilidades de composição e despertaram interesse geral. No entanto, há falta de procedimentos definidos e pensamento crítico em relação às diretrizes de mixing de áudio para som surround na música popular. Para abordar isso, foi conduzido um estudo empírico, analisando músicas premiadas na categoria de Melhor Álbum Imersivo do Grammy de 2005 até hoje. O objetivo era identificar tendências nas abordagens de mixing. Posição, trajetória e dinâmica foram consideradas como variáveis. O estudo usou uma matriz de alto-falantes múltiplos e codificação Ambisonic. Seções das músicas foram analisadas individualmente usando um diagrama de anotação radial proposto para coletar posições de instrumentos, trajetórias e dinâmica. Sete músicas foram selecionadas e analisadas em suas versões de mixing de som surround 5.1 e versão estéreo para comparação. Os resultados mostraram posições consistentes dos instrumentos em todas as músicas. Instrumentos rítmicos e o baixo foram colocados no centro. Os vocais principais tiveram uma distribuição mais ampla nos canais frontais. Instrumentos harmônicos, excluindo o baixo, foram posicionados em posições mais amplas. Instrumentos solo ocuparam os canais esquerdo, direito e central. As dinâmicas enfatizaram elementos importantes, com amplitudes altas para vocais principais e instrumentos solo, e dinâmicas mais baixas para camadas de harmonia e ambiente. As trajetórias foram raramente usadas. Os resultados indicam um pensamento predominante channel-based nas práticas de música popular imersiva. As posições dos instrumentos agruparam-se em ângulos específicos, sugerindo um viés influenciado pelo sistema channel-based. A simetria entre as fontes de áudio é ubíqua. Trajetórias e adoção mais ampla das dimensões espaciais oferecidas pelas tecnologias de áudio imersivo foram limitadas. Técnicas utilizadas para mixing de áudio imersivo são altamente derivadas de abordagens estéreo. Exceções e abordagens secundárias observadas na mixing apresentam oportunidades para explorações futuras. O estudo enfatiza a importância de aprender com expressões musicais diversas e design de som para informar as estratégias de mixing de áudio imersivo.Spatial audio has long been a focus in music production, driving creative strategies in post-production. The advent of immersive sound technologies has expanded composition possibilities and attracted mainstream interest. However, there is a lack of defined procedures and critical thinking regarding audio mixing guidelines for surround sound in popular music. To address this, an empirical study was conducted, analyzing award-winning songs in the Grammy category of Best Immersive Album from 2005 to the present. The aim was to identify trends in mixing approaches. Position, trajectory, and dynamics were considered as variables. The study employed a multi-speaker array and Ambisonic encoding. Sections of songs were analyzed individually using a proposed radial annotation diagram to collect instrument positions, trajectories, and dynamics. Seven songs were selected and analyzed both in their 5.1 surround sound mix version and stereo mix version for comparison. Results showed consistent instrument positions across songs. Rhythmic instruments and the bass were placed in the center. Lead vocals had a wider spread across the front channels. Harmonic instruments, excluding the bass, were positioned in wider positions, often spreading from their perceived positions. Solo instruments occupied the left, right, and center channels. Dynamics emphasized important elements, with high amplitudes for main vocals and solo instruments, and lower dynamics for harmony and ambiance layers. Trajectories were rarely used. The findings indicate a prevailing channel-based thinking in immersive popular music practices. Instrument positions clustered around specific angles, suggesting a bias influenced by the channel-based system. The symmetry between audio sources is ubiquitous. Trajectories and wider adoption of spatial dimensions offered by immersive audio technologies were limited. Techniques employed for immersive audio mixing are highly derivative of stereo approaches. Outliers and secondary approaches observed in the mixing present avenues for future exploration. The study emphasizes the importance of learning from diverse musical expressions and sound design to inform immersive audio-mixing strategies

Guitars with Ambisonic Spatial Performance (GASP): An immersive guitar system

The GASP project investigates the design and realisation of an Immersive Guitar System. It brings together a range of sound processing and spatialising technologies and applies them to a specific musical instrument ‒ the Electric Guitar. GASP is an ongoing innovative audio project, fusing the musical with the technical, combining the processing of each stringʼs output (which we called timbralisation) with spatial sound. It is also an artistic musical project, where space becomes a performance parameter, providing new experimental immersive sound production techniques for the guitarist and music producer. Several ways of reimagining the electric guitar as an immersive sounding instrument have been considered, the primary method using Ambisonics. However, additionally, some complementary performance and production techniques have emerged from the use of divided pickups, supporting both immersive live performance and studio post-production. GASP Live offers performers and audiences new real-time sonic-spatial perspectives, where the guitarist or a Live GASP producer can have real-time control of timbral, spatial, and other performance features, such as: timbral crossfading, switching of split-timbres across strings, spatial movement where Spatial Patterns may be selected and modulated, control of Spatial Tempo, and real-time performance re-tuning. For GASP recording and post-production, individual string note patterns may be visualised in Reaper DAW,2 from which, analyses and judgements can be made to inform post-production decisions for timbralisation and spatialisation. An appreciation of auditory grouping and perceptual streaming (Bregman, 1994) has informed GASP production ideas. For performance monitoring or recorded playback, the immersive audio would typically be heard over a circular array of loudspeakers, or over headphones with head-tracked binaural reproduction. This paper discusses the design of the system and its elements, investigates other applications of divided pickups, namely GASPʼs Guitarpeggiator, and reflects on productions made so far

Two-Dimensional Spatial Control using Multiple Circular Loudspeaker Arrays and Elliptical Loudspeaker Array

電気通信大学202

Guitars with Ambisonic Spatial Performance (GASP) An immersive guitar system

The GASP project investigates the design and realisation of an Immersive Guitar System. It brings together a range of sound processing and spatialising technologies and applies them to a specific musical instrument – the Electric Guitar. GASP is an ongoing innovative audio project, fusing the musical with the technical, combining the processing of each string’s output (which we called timbralisation) with spatial sound. It is also an artistic musical project, where space becomes a performance parameter, providing new experimental immersive sound production techniques for the guitarist and music producer. Several ways of reimagining the electric guitar as an immersive sounding instrument have been considered, the primary method using Ambisonics. However, additionally, some complementary performance and production techniques have emerged from the use of divided pickups, supporting both immersive live performance and studio post-production. GASP Live offers performers and audiences new real-time sonic-spatial perspectives, where the guitarist or a Live GASP producer can have real-time control of timbral, spatial, and other performance features, such as: timbral crossfading, switching of split-timbres across strings, spatial movement where Spatial Patterns may be selected and modulated, control of Spatial Tempo, and real-time performance re-tuning. For GASP recording and post-production, individual string note patterns may be visualised in Reaper DAW,2 from which, analyses and judgements can be made to inform post-production decisions for timbralisation and spatialisation. An appreciation of auditory grouping and perceptual streaming (Bregman, 1994) has informed GASP production ideas. For performance monitoring or recorded playback, the immersive audio would typically be heard over a circular array of loudspeakers, or over headphones with head-tracked binaural reproduction. This paper discusses the design of the system and its elements, investigates other applications of divided pickups, namely GASP’s Guitarpeggiator, and reflects on productions made so far

Spatial Multizone Soundfield Reproduction Design

It is desirable for people sharing a physical space to access different multimedia information streams simultaneously. For a good user experience, the interference of the different streams should be held to a minimum. This is straightforward for the video component but currently difficult for the audio sound component. Spatial multizone soundfield reproduction, which aims to provide an individual sound environment to each of a set of listeners without the use of physical isolation or headphones, has drawn significant attention of researchers in recent years. The realization of multizone soundfield reproduction is a conceptually challenging problem as currently most of the soundfield reproduction techniques concentrate on a single zone. This thesis considers the theory and design of a multizone soundfield reproduction system using arrays of loudspeakers in given complex environments. We first introduce a novel method for spatial multizone soundfield reproduction based on describing the desired multizone soundfield as an orthogonal expansion of formulated basis functions over the desired reproduction region. This provides the theoretical basis of both 2-D (height invariant) and 3-D soundfield reproduction for this work. We then extend the reproduction of the multizone soundfield over the desired region to reverberant environments, which is based on the identification of the acoustic transfer function (ATF) from the loudspeaker over the desired reproduction region using sparse methods. The simulation results confirm that the method leads to a significantly reduced number of required microphones for an accurate multizone sound reproduction compared with the state of the art, while it also facilitates the reproduction over a wide frequency range. In addition, we focus on the improvements of the proposed multizone reproduction system with regard to practical implementation. The so-called 2.5D multizone oundfield reproduction is considered to accurately reproduce the desired multizone soundfield over a selected 2-D plane at the height approximately level with the listener’s ears using a single array of loudspeakers with 3-D reverberant settings. Then, we propose an adaptive reverberation cancelation method for the multizone soundfield reproduction within the desired region and simplify the prior soundfield measurement process. Simulation results suggest that the proposed method provides a faster convergence rate than the comparative approaches under the same hardware provision. Finally, we conduct the real-world implementation based on the proposed theoretical work. The experimental results show that we can achieve a very noticeable acoustic energy contrast between the signals recorded in the bright zone and the quiet zone, especially for the system implementation with reverberation equalization

Music in Site: Integrating elements of site-specificity into composition

Architectural spaces and their acoustic characteristics offer unique musical material for the compositional process. Acoustic and physical design features of unorthodox performance spaces can become part of works and their performances. This thesis examines ways to integrate acoustic characteristics of an architectural space into the compositional process, and discusses how different levels of site-specificity may be engaged in this process. This research grew from an interest in composing music for the acoustic problems of performance spaces rather than trying to resist them, after a jazz ensemble performance in a large reverberant space. This led to exploring built environments that offered an acoustic characteristic which could be used to initiate musical material which is directly linked to the site. Three sites were chosen as starting points for composition according to their varying acoustic characteristics; a stairwell, a tunnel and a bridge. Each site presented unique acoustic and physical characteristics as well as challenges which required creating a precompositional testing and work-shopping methodology. The processes and experiments engaged led to three varying compositions which are discussed in part two of this exegesis. The research also draws inspiration from secondary literature in theatre, dance and choreography that interrogates the way works can be linked to their particular site. British academic Fiona Wilkie developed a scale of site-specificity for theatre that provides a useful tool to gauge the level and type of site interaction each composition maintains and forms a frame for the different approaches used. In addition, dance choreographer and theorist, Fiona Hunter’s methodology for testing the possibilities of a site for an artwork has been employed. The three creative works at the centre of this project, Stairwell to Fifteen (four brass musicians, cimbalom and found percussive sounds in a stairwell), From Traffic Rises (eight acoustic musicians, electronics and four speakers) and Tunnel Listen (two clarinets, soprano sax, alto sax, tenor sax, two trumpets, two trombones and tuba) explore a range of approaches to site-specificity and embodiment as compositional devices. The outcome of this research has not only been the creation of these three new works, but also the exploration of an alternative compositional process which begins and is informed by a physical space as a musical starting point

Music as complex emergent behaviour : an approach to interactive music systems

Access to the full-text thesis is no longer available at the author's request, due to 3rd party copyright restrictions. Access removed on 28.11.2016 by CS (TIS).Metadata merged with duplicate record (http://hdl.handle.net/10026.1/770) on 20.12.2016 by CS (TIS).This is a digitised version of a thesis that was deposited in the University Library. If you are the author please contact PEARL Admin ([email protected]) to discuss options.This thesis suggests a new model of human-machine interaction in the domain of non-idiomatic musical improvisation. Musical results are viewed as emergent phenomena issuing from complex internal systems behaviour in relation to input from a single human performer. We investigate the prospect of rewarding interaction whereby a system modifies itself in coherent though non-trivial ways as a result of exposure to a human interactor. In addition, we explore whether such interactions can be sustained over extended time spans. These objectives translate into four criteria for evaluation; maximisation of human influence, blending of human and machine influence in the creation of machine responses, the maintenance of independent machine motivations in order to support machine autonomy and finally, a combination of global emergent behaviour and variable behaviour in the long run. Our implementation is heavily inspired by ideas and engineering approaches from the discipline of Artificial Life. However, we also address a collection of representative existing systems from the field of interactive composing, some of which are implemented using techniques of conventional Artificial Intelligence. All systems serve as a contextual background and comparative framework helping the assessment of the work reported here. This thesis advocates a networked model incorporating functionality for listening, playing and the synthesis of machine motivations. The latter incorporate dynamic relationships instructing the machine to either integrate with a musical context suggested by the human performer or, in contrast, perform as an individual musical character irrespective of context. Techniques of evolutionary computing are used to optimise system components over time. Evolution proceeds based on an implicit fitness measure; the melodic distance between consecutive musical statements made by human and machine in relation to the currently prevailing machine motivation. A substantial number of systematic experiments reveal complex emergent behaviour inside and between the various systems modules. Music scores document how global systems behaviour is rendered into actual musical output. The concluding chapter offers evidence of how the research criteria were accomplished and proposes recommendations for future research

Effects of errorless learning on the acquisition of velopharyngeal movement control

Session 1pSC - Speech Communication: Cross-Linguistic Studies of Speech Sound Learning of the Languages of Hong Kong (Poster Session)The implicit motor learning literature suggests a benefit for learning if errors are minimized during practice. This study investigated whether the same principle holds for learning velopharyngeal movement control. Normal speaking participants learned to produce hypernasal speech in either an errorless learning condition (in which the possibility for errors was limited) or an errorful learning condition (in which the possibility for errors was not limited). Nasality level of the participants’ speech was measured by nasometer and reflected by nasalance scores (in %). Errorless learners practiced producing hypernasal speech with a threshold nasalance score of 10% at the beginning, which gradually increased to a threshold of 50% at the end. The same set of threshold targets were presented to errorful learners but in a reversed order. Errors were defined by the proportion of speech with a nasalance score below the threshold. The results showed that, relative to errorful learners, errorless learners displayed fewer errors (50.7% vs. 17.7%) and a higher mean nasalance score (31.3% vs. 46.7%) during the acquisition phase. Furthermore, errorless learners outperformed errorful learners in both retention and novel transfer tests. Acknowledgment: Supported by The University of Hong Kong Strategic Research Theme for Sciences of Learning © 2012 Acoustical Society of Americapublished_or_final_versio

Optimization and improvements in spatial sound reproduction systems through perceptual considerations

[ES] La reproducción de las propiedades espaciales del sonido es una cuestión cada vez más importante en muchas aplicaciones inmersivas emergentes. Ya sea en la reproducción de contenido audiovisual en entornos domésticos o en cines, en sistemas de videoconferencia inmersiva o en sistemas de realidad virtual o aumentada, el sonido espacial es crucial para una sensación de inmersión realista. La audición, más allá de la física del sonido, es un fenómeno perceptual influenciado por procesos cognitivos. El objetivo de esta tesis es contribuir con nuevos métodos y conocimiento a la optimización y simplificación de los sistemas de sonido espacial, desde un enfoque perceptual de la experiencia auditiva. Este trabajo trata en una primera parte algunos aspectos particulares relacionados con la reproducción espacial binaural del sonido, como son la escucha con auriculares y la personalización de la Función de Transferencia Relacionada con la Cabeza (Head Related Transfer Function - HRTF). Se ha realizado un estudio sobre la influencia de los auriculares en la percepción de la impresión espacial y la calidad, con especial atención a los efectos de la ecualización y la consiguiente distorsión no lineal. Con respecto a la individualización de la HRTF se presenta una implementación completa de un sistema de medida de HRTF y se introduce un nuevo método para la medida de HRTF en salas no anecoicas. Además, se han realizado dos experimentos diferentes y complementarios que han dado como resultado dos herramientas que pueden ser utilizadas en procesos de individualización de la HRTF, un modelo paramétrico del módulo de la HRTF y un ajuste por escalado de la Diferencia de Tiempo Interaural (Interaural Time Difference - ITD). En una segunda parte sobre reproducción con altavoces, se han evaluado distintas técnicas como la Síntesis de Campo de Ondas (Wave-Field Synthesis - WFS) o la panoramización por amplitud. Con experimentos perceptuales se han estudiado la capacidad de estos sistemas para producir sensación de distancia y la agudeza espacial con la que podemos percibir las fuentes sonoras si se dividen espectralmente y se reproducen en diferentes posiciones. Las aportaciones de esta investigación pretenden hacer más accesibles estas tecnologías al público en general, dada la demanda de experiencias y dispositivos audiovisuales que proporcionen mayor inmersión.[CA] La reproducció de les propietats espacials del so és una qüestió cada vegada més important en moltes aplicacions immersives emergents. Ja siga en la reproducció de contingut audiovisual en entorns domèstics o en cines, en sistemes de videoconferència immersius o en sistemes de realitat virtual o augmentada, el so espacial és crucial per a una sensació d'immersió realista. L'audició, més enllà de la física del so, és un fenomen perceptual influenciat per processos cognitius. L'objectiu d'aquesta tesi és contribuir a l'optimització i simplificació dels sistemes de so espacial amb nous mètodes i coneixement, des d'un criteri perceptual de l'experiència auditiva. Aquest treball tracta, en una primera part, alguns aspectes particulars relacionats amb la reproducció espacial binaural del so, com són l'audició amb auriculars i la personalització de la Funció de Transferència Relacionada amb el Cap (Head Related Transfer Function - HRTF). S'ha realitzat un estudi relacionat amb la influència dels auriculars en la percepció de la impressió espacial i la qualitat, dedicant especial atenció als efectes de l'equalització i la consegüent distorsió no lineal. Respecte a la individualització de la HRTF, es presenta una implementació completa d'un sistema de mesura de HRTF i s'inclou un nou mètode per a la mesura de HRTF en sales no anecoiques. A mès, s'han realitzat dos experiments diferents i complementaris que han donat com a resultat dues eines que poden ser utilitzades en processos d'individualització de la HRTF, un model paramètric del mòdul de la HRTF i un ajustament per escala de la Diferencià del Temps Interaural (Interaural Time Difference - ITD). En una segona part relacionada amb la reproducció amb altaveus, s'han avaluat distintes tècniques com la Síntesi de Camp d'Ones (Wave-Field Synthesis - WFS) o la panoramització per amplitud. Amb experiments perceptuals, s'ha estudiat la capacitat d'aquests sistemes per a produir una sensació de distància i l'agudesa espacial amb que podem percebre les fonts sonores, si es divideixen espectralment i es reprodueixen en diferents posicions. Les aportacions d'aquesta investigació volen fer més accessibles aquestes tecnologies al públic en general, degut a la demanda d'experiències i dispositius audiovisuals que proporcionen major immersió.[EN] The reproduction of the spatial properties of sound is an increasingly important concern in many emerging immersive applications. Whether it is the reproduction of audiovisual content in home environments or in cinemas, immersive video conferencing systems or virtual or augmented reality systems, spatial sound is crucial for a realistic sense of immersion. Hearing, beyond the physics of sound, is a perceptual phenomenon influenced by cognitive processes. The objective of this thesis is to contribute with new methods and knowledge to the optimization and simplification of spatial sound systems, from a perceptual approach to the hearing experience. This dissertation deals in a first part with some particular aspects related to the binaural spatial reproduction of sound, such as listening with headphones and the customization of the Head Related Transfer Function (HRTF). A study has been carried out on the influence of headphones on the perception of spatial impression and quality, with particular attention to the effects of equalization and subsequent non-linear distortion. With regard to the individualization of the HRTF a complete implementation of a HRTF measurement system is presented, and a new method for the measurement of HRTF in non-anechoic conditions is introduced. In addition, two different and complementary experiments have been carried out resulting in two tools that can be used in HRTF individualization processes, a parametric model of the HRTF magnitude and an Interaural Time Difference (ITD) scaling adjustment. In a second part concerning loudspeaker reproduction, different techniques such as Wave-Field Synthesis (WFS) or amplitude panning have been evaluated. With perceptual experiments it has been studied the capacity of these systems to produce a sensation of distance, and the spatial acuity with which we can perceive the sound sources if they are spectrally split and reproduced in different positions. The contributions of this research are intended to make these technologies more accessible to the general public, given the demand for audiovisual experiences and devices with increasing immersion.Gutiérrez Parera, P. (2020). Optimization and improvements in spatial sound reproduction systems through perceptual considerations [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/142696TESI