Why would Musical Training Benefit the Neural Encoding of Speech? The OPERA Hypothesis

Mounting evidence suggests that musical training benefits the neural encoding of speech. This paper offers a hypothesis specifying why such benefits occur. The “OPERA” hypothesis proposes that such benefits are driven by adaptive plasticity in speech-processing networks, and that this plasticity occurs when five conditions are met. These are: (1) Overlap: there is anatomical overlap in the brain networks that process an acoustic feature used in both music and speech (e.g., waveform periodicity, amplitude envelope), (2) Precision: music places higher demands on these shared networks than does speech, in terms of the precision of processing, (3) Emotion: the musical activities that engage this network elicit strong positive emotion, (4) Repetition: the musical activities that engage this network are frequently repeated, and (5) Attention: the musical activities that engage this network are associated with focused attention. According to the OPERA hypothesis, when these conditions are met neural plasticity drives the networks in question to function with higher precision than needed for ordinary speech communication. Yet since speech shares these networks with music, speech processing benefits. The OPERA hypothesis is used to account for the observed superior subcortical encoding of speech in musically trained individuals, and to suggest mechanisms by which musical training might improve linguistic reading abilities

Musical timbre: bridging perception with semantics

Musical timbre is a complex and multidimensional entity which provides information regarding the properties of a sound source (size, material, etc.). When it comes to music, however, timbre does not merely carry environmental information, but it also conveys aesthetic meaning. In this sense, semantic description of musical tones is used to express perceptual concepts related to artistic intention. Recent advances in sound processing and synthesis technology have enabled the production of unique timbral qualities which cannot be easily associated with a familiar musical instrument. Therefore, verbal description of these qualities facilitates communication between musicians, composers, producers, audio engineers etc. The development of a common semantic framework for musical timbre description could be exploited by intuitive sound synthesis and processing systems and could even influence the way in which music is being consumed. This work investigates the relationship between musical timbre perception and its semantics. A set of listening experiments in which participants from two different language groups (Greek and English) rated isolated musical tones on semantic scales has tested semantic universality of musical timbre. The results suggested that the salient semantic dimensions of timbre, namely: luminance, texture and mass, are indeed largely common between these two languages. The relationship between semantics and perception was further examined by comparing the previously identified semantic space with a perceptual timbre space (resulting from pairwise dissimilarity rating of the same stimuli). The two spaces featured a substantial amount of common variance suggesting that semantic description can largely capture timbre perception. Additionally, the acoustic correlates of the semantic and perceptual dimensions were investigated. This work concludes by introducing the concept of partial timbre through a listening experiment that demonstrates the influence of background white noise on the perception of musical tones. The results show that timbre is a relative percept which is influenced by the auditory environment

Perceptual and physiological measures of auditory selective attention in normal-hearing and hearing-impaired listeners

Human listeners can direct top-down spatial auditory attention to listen selectively to one sound source amidst competing sounds. However, many listeners with hearing loss (HL) have trouble on tasks requiring selective auditory attention; even listeners with normal hearing thresholds (NHTs) differ in this ability. Selective attention depends on both top-down executive control and coding fidelity of the peripheral auditory system. Here we explore how low-level sensory perception and high-level attentional modulation interact to contribute to auditory selective attention for listeners with NHTs and HL. In the first study, we designed a paradigm to allow simultaneous measurement of envelope following responses (EFRs), onset event-related potentials (ERPs), and behavioral performance. We varied conditions to alter the degree to which the bottleneck limiting behavior was due to the coding of fine stimulus details vs. top-down control of attentional focus. We found attention modulated ERPs, from cortex, but not EFRs from the brainstem. Importantly, when coding fidelity limited the task, EFRs but not ERPs correlated with behavior; conversely, when sensory cues for segregation were robust, individual behavior correlated with both EFR strength and strength of attentional modulation of cortical responses. In the second study, we explored how HL affects control of auditory selective attention. Listeners with NHTs or with HL identified a simple melody presented simultaneously with two competing melodies, each from different spatial locations. Compared to NHT listeners, HL listeners both performed more poorly and showed less robust attentional modulation of cortical ERPs. While both groups showed some cortical suppression of distracting streams, this modulation was weaker in HL listeners, especially when spatial separation between attended and distracting streams was small. In the final study, we compared temporal coding precision in listeners with NHT and HL using both behavioral and physiological measures. We found that listeners with HL are more sensitive than listeners with NHT to amplitude modulation in both measures. Within the NHT listener group, we found a strong correlation between behavioral and electrophysiological measurements, consistent with cochlear synaptopathy. Overall, these studies demonstrate that everyday communication abilities depend jointly on both low-level differences in sensory coding and high-level ability to control attention

A primate model of human cortical analysis of auditory objects

PhD ThesisThe anatomical organization of the auditory cortex in old world monkeys is similar to that in humans. But how good are monkeys as a model of human cortical analysis of auditory objects? To address this question I explore two aspects of auditory objectprocessing: segregation and timbre. Auditory segregation concerns the ability of animals to extract an auditory object of relevance from a background of competing sounds. Timbre is an aspect of object identity distinct from pitch. In this work, I study these phenomena in rhesus macaques using behaviour and functional magnetic resonance imaging (fMRI). I specifically manipulate one dimension of timbre, spectral flux: the rate of change of spectral energy.I present this thesis in five chapters. Chapter 1 presents background on auditory processing, macaque auditory cortex, models of auditory segregation, and dimensions of timbre. Chapter 2 presents an introduction to fMRI, the design of the fMRI experiments and analysis of fMRI data, and macaque behavioural training techniques employed. Chapter 3 presents results from the fMRI and behavioural experiments on macaques using a stochastic figure-ground stimulus. Chapter 4 presents the results from the fMRI experiment in macaques using spectral flux stimulus. Chapter 5 concludes with a general discussion of the results from both the studies and some future directions for research.In summary, I show that there is a functional homology between macaques and humans in the cortical processing of auditory figure-ground segregation. However, there is no clear functional homology in the processing of spectral flux between these species. So I conclude that, despite clear similarities in the organization of the auditory cortex and processing of auditory object segregation, there are important differences in how complex cues associated with auditory object identity are processed in the macaque and human auditory brains.Wellcome Trust U

Interactive real-time musical systems

PhDThis thesis focuses on the development of automatic accompaniment systems. We investigate previous systems and look at a range of approaches that have been attempted for the problem of beat tracking. Most beat trackers are intended for the purposes of music information retrieval where a `black box' approach is tested on a wide variety of music genres. We highlight some of the diffculties facing offline beat trackers and design a new approach for the problem of real-time drum tracking, developing a system, B-Keeper, which makes reasonable assumptions on the nature of the signal and is provided with useful prior knowledge. Having developed the system with offline studio recordings, we look to test the system with human players. Existing offline evaluation methods seem less suitable for a performance system, since we also wish to evaluate the interaction between musician and machine. Although statistical data may reveal quantifiable measurements of the system's predictions and behaviour, we also want to test how well it functions within the context of a live performance. To do so, we devise an evaluation strategy to contrast a machine-controlled accompaniment with one controlled by a human. We also present recent work on a real-time multiple pitch tracking, which is then extended to provide automatic accompaniment for harmonic instruments such as guitar. By aligning salient notes in the output from a dual pitch tracking process, we make changes to the tempo of the accompaniment in order to align it with a live stream. By demonstrating the system's ability to align offline tracks, we can show that under restricted initial conditions, the algorithm works well as an alignment tool

Inferring rules from sound: The role of domain-specific knowledge in speech and music perception

Speech and music are two forms of complex auditory structure that play fundamental roles in everyday human experience and require certain basic perceptual and cognitive abilities. Nevertheless, when attempting to infer patterns from sequential auditory input, human listeners may use the same information differently depending on whether a sound is heard in a linguistic vs. musical context. The goal of these studies was to examine the role of domain-specific knowledge in auditory pattern perception. Specifically, the study examined the inference of rules in novel sound sequences that contained patterns of spectral structure (speech or instrument timbre) and fundamental frequency (pitch). Across all experiments, participants were first familiarized to a sequence containing pitch or syllable patterns that followed a particular rule (e.g., ABA), and they were subsequently asked to rate the similarity of novel sequences that were consistent or inconsistent with that rule. In two experiments participants were familiarized to either a pitch or syllable rule, and in a third experiment they were familiarized to simultaneous conflicting rules (e.g. pitch following ABA but syllables following ABB). Although participants readily detected inconsistent stimuli after familiarization to a single rule, in the conflicting two-rule condition they gave high similarity ratings to any stimulus that obeyed the syllable rule, regardless of whether or not the music dimension was consistent or inconsistent with familiarization. Three additional experiments took the same approach but tested rule-learning on the basis of pitch or timbre (instrument) sequences. In these experiments, participants gave the highest similarity ratings when the pitch rule was preserved, regardless of variation in the timbre dimension. These results support the notion that adults filter information according to domain-specific knowledge and expectations, presumably due to perceptual learning processes that take place during early development

The effects of rhythmic structure on tapping accuracy

Prior investigations of simple rhythms in familiar time signatures have shown the importance of several mechanisms; notably, those related to metricization and grouping. But there has been limited study of complex rhythms, including those in unfamiliar time signatures, such as are found outside mainstream Western music. Here, we investigate how the structures of 91 rhythms with nonisochronous onsets (mostly complex, several in unfamiliar time signatures) influence the accuracy, velocity, and timing of taps made by participants attempting to synchronize with these onsets. The onsets were piano-tone cues sounded at a well-formed subset of isochronous cymbal pulses; the latter occurring every 234 ms. We modelled tapping at both the rhythm level and the pulse level; the latter provides insight into how rhythmic structure makes some cues easier to tap and why incorrect (uncued) taps may occur. In our models, we use a wide variety of quantifications of rhythmic features, several of which are novel and many of which are indicative of underlying mechanisms, strategies, or heuristics. The results show that, for these tricky rhythms, taps are disrupted by unfamiliar period lengths and are guided by crude encodings of each rhythm: the density of rhythmic cues, their circular mean and variance, and recognizing common small patterns and the approximate positions of groups of cues. These lossy encodings are often counterproductive for discriminating between cued and uncued pulses and are quite different to mechanisms—such as metricization and emphasizing group boundaries—thought to guide tapping behaviours in learned and familiar rhythms

Research into the practical and policy applications of soundscape concepts and techniques in urban areas

Executive Summary 1 The aim of this review was to investigate existing research into soundscape concepts and to produce recommendations for future research into the practical identification, management and enhancement of soundscapes in urban areas. 2 Existing research on soundscapes was investigated using four methods: i. a survey of more than 500 papers in the academic literature, ii. an analysis of 27 case studies of soundscape assessment, iii. an analysis of 15 case studies of soundscape design, and iv. interviews with five key soundscape experts. Analysis of this data was conducted to identify significant gaps in the knowledge base and suggest a way of obtaining a practical soundscape assessment method. 3 Soundscapes were found to be a highly multi-disciplinary topic, with many different ideas, concepts, aims and methods evident in the literature. The definition of the term soundscape is itself not settled; for the purposes of this project, we have defined it as “the totality of all sounds within a location with an emphasis on the relationship between individual’s or society’s perception of, understanding of and interaction with the sonic environment.” 4 This review highlights that a range of methodological approaches have been used to establish classifications and categorisations of sounds and soundscapes. The relationship between different categories of sounds and their interaction needs to be considered to increase the understanding of soundscape assessments and to derive soundscape classifications. 5 The different methods and tools used to assess soundscapes, in a variety of locations, each have advantages and disadvantages; using a number of methods in one case study can help to mitigate against the disadvantages of any one method. The case studies assessed in this report demonstrate the importance of individual and demographic similarities/differences, people’s behaviour, physical aspects of the soundscape, other sensory and environmental elements, and the general location and context, in understanding and assessing soundscapes. 6 Soundscape assessments involving a subjective component have highlighted a number of variables that play a part in the assessment. These include the individual’s knowledge and prior experience of the soundscape, the meaning they derive from it, their attitude towards the sound source, their behaviour, their noise sensitivity, demographic and cultural dimensions, and their sense of control over the noise. 7 Research has shown that sometimes a soundscape is perceived as a collection of the individual sounds of which it is comprised; soundscape assessments are therefore related to the assessment of those sound types. This implies that soundscape assessment relies upon the identification of the sounds, the prominence of the sounds, and potentially the ratio of certain sound types to other sound types within the soundscape. It is also highlighted that, because the soundscape varies over time, note must be taken of the fact that any soundscape assessment relates to a singular moment in time. Furthermore, research has shown that soundscape assessments can be dependent on an individual’s memory (when using subjective assessments methods) and/or the segment of the soundscape that was recorded (when playing back recorded soundscapes in a laboratory situation). 8 Multi-sensory experience is also shown to be highly relevant to soundscape assessment and must therefore be acknowledged as soundscapes are not perceived in sensory isolation; in particular audio-visual interactions have been shown to have an effect on soundscape perception. Many researchers point to the importance of understanding the full environmental and social context for soundscape assessment, the relevance of comparing similar place types, and the effect of moving between one soundscape and another on an assessment. 9 Turning to the subject of soundscape design, it is noted that there is a dearth of case studies involving the modification and design of soundscapes, both in the UK and internationally. The rationale behind many of the case studies’ focus upon or consideration of sound was the improvement of a soundscape that was negatively affected by the sound of traffic. Approaches to soundscape design varied, ranging from the use of noise control elements, such as barriers and absorbers, to the utilisation or exploitation of natural elements that already exist in the location. Some case studies introduced sounds to the soundscape, in particular water sounds, while others incorporated specific sonic art installations to alter the soundscape or detract attention from existing features of the soundscape. A number of case studies used design alterations to improve the soundscape and perception of the soundscape including altering visual aspects of the place, altering the layout of the area, pedestrianisation of the area, and providing entertainment facilities (e.g. cafes). 10 Case studies whereby design modifications or interventions have taken place, have had little or no formal evaluation of their success. The studies that were evaluated used a number of different methods involving both objective and subjective measures and included the experimental comparison of subjective ratings, observations of people’s behaviour, recognition and awards for good designs, and level of complaints about the soundscape. This demonstrates that different evaluation tools may be necessary dependent upon the type of soundscape intervention being evaluated. Additionally, by combining methods to produce an interdisciplinary evaluation, a more accurate understanding of the success of the soundscape design is possible, hence improving future interventions. 11 The relationship between environment and individual is complex, with many factors, some of which cannot currently be quantified. Important factors include: prominent individual sound sources, the interaction of sources, other sensory stimuli and contextual and individual factors such as meaning, and expectation. Some of these factors can be captured by subjective rating scales for high-level concepts like ‘calmness’, ‘vibrancy’ and ‘spaciousness’. Other factors, such as the semantic meaning of a soundscape are best characterised currently by qualitative descriptors. There are good prospects for developing objective acoustic metrics to evaluate some factors but in most cases this work is still at an early stage, and the methods developed so far have only been applied in specific contexts; to provide metrics that are more broadly applicable they would need to be evaluated in a broader range of locations and conditions. The expert interviews and case studies illustrated the diversity of views across different disciplines on the most promising soundscape methods. All the interviewees agreed on the need for an interdisciplinary approach, and on the need to retain some form of subjective rating when assessing soundscapes. 12 Ultimately, six important gaps have been identified in the soundscape knowledge base. These are areas where more research would significantly improve understanding soundscape assessments. These gaps have been identified as: i. a lack of genuinely interdisciplinary projects (characterised by a shared perspective) instead of multidisciplinary projects (where researchers work in parallel within their own disciplines). These are needed to deal with the multidimensional experience of soundscape perception. ii. a lack of basic knowledge on many aspects of soundscape cognition, perception and classification. iii.a need for large-scale robust field trials of soundscape assessment methods instead of the more common experiment of a new method in a single location. iv. a need to develop more soundscape-specific indicators and tools that could eventually be used for soundscape design. v. a need to rigorously assess deliberate soundscape interventions to understand which design aspects work and which do not. vi. a lack of a close connection between soundscape research, design and planning practice. 13 Finally, a new research project is proposed to develop a robust field assessment method. The aim of this project is to develop a method based on existing research methods but introducing greater confidence by trialling the method across many real urban soundscapes. Options are presented for developing a purely qualitative assessment tool or one that incorporates and integrates both qualitative and quantitative ratings. 14 We recommend that a first step for an assessment method, which could realistically be developed in the near future, should be based on qualitative methods. A second iteration of this soundscape assessment tool could supplement the qualitative techniques with quantitative methods, first based on subjective rating scales and eventually on objective metrics which predict the subjective ratings