Interactive performance for musicians with a hearing impairment

How can we perceive music if we cannot hear it properly? The achievements of deaf musicians suggest it is possible not only to perceive music, but to perform with other musicians. Yet very little research exists to explain how this is possible. This thesis addresses this problem and explores the premise that vibrations felt on the skin may facilitate interactive music making. An initial interview study found that, while vibrations are sometimes perceived, it is predominantly the use of visual and physical cues that are relied upon in group performance to help stay in time and in tune with other players. The findings informed the design of two observation studies exploring the effects of i) artificial attenuation of auditory information and ii) natural deafness on performance behaviours. It was shown that profound congenital deafness affected the players’ movements and their gazes/glances towards each other while mild or moderate levels of attenuation or deafness did not. Nonetheless, all players, regardless of hearing level, reciprocated the behaviours of co-performers suggesting the influence of social factors benefitting verbal and non-verbal communication between players. Finally, a series of three psychophysical experiments was designed to explore the perception of pitch on the skin using vibrations. The first study found that vibrotactile detection thresholds were not affected by hearing impairments. The second established that the relative pitches of intervals larger than a major 6th were easy to discriminate, but this was not possible for semitones. The third showed that tones an octave apart could be memorised and identified accurately, but were confused when less than a perfect 4th apart. The thesis concludes by evaluating the potential of vibrotactile technology to facilitate interactive performance for musicians with hearing impairments. By considering the psychophysical, behavioural and qualitative data together, it is suggested that signal processing strategies in vibrotactile technology should take social, cognitive and perceptual factors into account

The Human Faculty for Music: What's special about it?

Abstract (short version) This thesis presents a model of a narrow faculty for music - qualities that are at once universally present and operational in music across cultures whilst also being specific to our species and to the domain of music. The comparative approach taken focuses on core psychological and physiological capabilities that root and enable appropriate engagement with music rather than on their observable physical correlates. Configurations of musical pulse; musical tone; and musical motivation are described as providing a sustained attentional structure for managing personal experience and interpersonal interaction and as offering a continually renewing phenomenological link between the immediate past, the perceptual present and future expectation. Constituent parts of the narrow faculty for music are considered most fundamentally as a potentiating, quasi-architectural framework in which our most central affective and socio-intentional drives are afforded extended time, stability, and a degree of abstraction, intensity, focus and meaning. The author contends, therefore, that music's defining characteristics, specific functionalities and/or situated efficacies are not demarcated in broadly termed “musical” qualities such as melodic contour or rhythm or in those surprisingly elusive “objective facts” of musical structure. Rather they are solely the attentional/motivational frameworks which root our faculty to make and make sense of music. Our generic capacities for culture and the manifold uses of action, gesture, and sound to express and induce emotion; to regulate affective states; to create or reflect meaning; to signify; to ritualize; coordinate; communicate; interrelate; embody; entrain; and/or intentionalize, none of these is assessed as being intrinsically unique to music performance. Music is, instead, viewed as an ordered expression of human experience, behaviour, interaction, and vitality, all shaped, shared, given significance, and/or transformed in time. The relevance of this model to topical debates on music and evolution is discussed and the author contends that the perspective offered affords significant implications for our understanding of why music is evidently and remarkably effective in certain settings and in the pursuit of certain social, individual, and therapeutic goals.Cambridge University Millennium Fun

Strategies for Environmental Sound Measurement, Modelling, and Evaluation

This thesis is a portfolio of research into three aspects of environmental sound: its measurement, modelling, and evaluation. In each of these areas, this body of work aims to make use of soundscape methodologies in order to develop an understanding of different aspects of our relationship with our sonic environments. This approach is representative of the nature of soundscape research, which makes use of elements of many other research areas, including acoustics, psychology, sociology, and musicology. The majority of prior acoustic measurement research has considered indoor recording, often of music, and measurement of acoustic parameters of indoor spaces such as concert halls and other performance spaces. One strand of this research has investigated how best to apply such techniques to the recording of environmental sound, and to the measurement of the acoustic impulse responses of outdoor spaces. Similarly, the majority of prior work in the field of acoustic modelling has also focussed mainly on indoor spaces. Presented here is the Waveguide Web, a novel method for the acoustic modelling of sparsely reflecting outdoor spaces. In the field of sound evaluation of sound, recent years have seen the development of soundscape techniques for the subjective rating of environmental sound, allowing for a better understanding of our relationship with our sonic surroundings. Research presented in this thesis has focussed on how best to improve these approaches in a suitably robust and intuitive manner, including the integration of visual stimuli in order to investigate the multi-modal perception of our surroundings. The aim of this thesis in making contributions to these three fields of environmental sound research is, in part, to highlight the importance of developing a comprehensive understanding of our sonic environments. Such an understanding could ultimately lead to the alleviation of noise problems, encourage greater engagement with environmental sound in the wider population, and allow for the design of more positive, restorative, soundscapes

Interaction in musical time

Social cognition in general, and rhythmic entrainment in particular, have previously mainly been studied in settings where isolated individuals perform controlled tasks. Recently, a number of alternative approaches have been developed to redefine what constitutes the “cognitive system”. In addition to the individual mind/brain, the body, the social context, and musical instruments should be included in the analytical framework. In general, this means studying cognition and behaviour at settings that are as naturalistic as possible. Music and dance are ideal domains for studying these phenomena, as naturally social, embodied activities. Extending the traditional setting poses many challenges. I make the case for focusing the analysis on the interaction of multiple participants, instead of trying to measure the performance or mind-states of the individuals. This interactionist approach requires a specific set of analysis tools. For this purpose, I distinguish rhytmic synchronisation from entrainment between mutually cooperative individuals. I discuss a range of options from circular statistics to cross-recurrence analysis to various correlation-based analyses. Through a number of pilot studies, I developed a cooperative tapping setup for studying rhythmic entrainment in dyads. Using this setup, I compared human–human interaction to synchronising with a computer. Surprisingly, two human tappers reached better synchronicity than a human with a computer tapper, even though the human pairs drifted in tempo. This demonstrated the power of mutual adaptation. In a second series of experiments, motion capture was used to investigate the embodied nature of rhythmic entrainment. These cross-cultural studies on African dance, illustrated in more detail how synchronicity was achieved through a process of continuous, mutual adaptation. We observed interesting contrasts in how Finnish novices and South-African or Kenyan experts exhibited embodied metrical structures. As a conclusion, mutual adaptation is a powerful and ubiquitous phenomenon that can only be observed in real-time interactions. It is a good example of the kind of “new psychology” that can be uncovered by adopting a social, embodied, and dynamic approach to cognition

Legibility of Musical Scores and Parallels with Language Reading

Following on the extensive literature within experimental psychology on the reading of natural language texts, I have undertaken a series of experiments on the sight-reading of musical scores that have shown that spacing of information, the structuring of the musical discourse, and the predictability of design in a score can aid its legibility in a manner similar to what has been shown in the language domain. Cultural studies of reading —particularly the works of Saenger— point in the same direction; according to these, the change in Medieval textual scripts from scriptura continua at the beginning of the eight century to the adoption of canonical separations between words, phrases, or paragraphs (which had fully spread throughout Europe by the mid-fourteenth century) significantly decreased the cognitive load and time that had previously been needed to decode a script. Crucially, this eliminated the need for the ancient techniques of the praelectio (initial decoding of the text by reading it aloud) and rote repetition for its comprehension, triggering a whole new culture of private fluent reading. Equally, the literature on music sight-reading (although lacking in systematic research based on objective measurements of legibility of texts) has proposed, based on surveys and studies of expertise, a series of cognitive models of the activity that prime, as factors that distinguish proficient readers from beginners: the integration of discursive elements into higher-order meaning units, the ability to predict upcoming information, and the awareness of the structuring of the text. The experiments reported here compared readings using conventional scores with readings using novel scores where the suggested advantages of information separation, integration and predictability were implemented in the design. Fluency of performance was measured primarily in terms of numbers of mistakes, results showing that readers played more accurately with the novel scores. Other, more qualitative, measurements —such as spectrogram coding of tempo stability, blind expert judgment of performance quality, and participant self-assessments— all showed strong positive correlations with the measurements of numbers of mistakes, with the novel scores producing performances that were more fluent and ranked as more trustworthy and musically satisfactory by experts and readers alike. These results will still need to be extrapolated to many other musical practices, but they serve to open a debate on the conventions of music publishing as they stand, and are well placed to open new lines of research in score legibility and design.Cambridge Home and EU Scholarship Scheme (CHESS