Prosody beyond pitch and emotion in speech and music: evidence from right hemisphere brain damage and congenital amusia

This dissertation examines the relationship of prosodic processing in language and music from a new perspective, considering acoustic features that have not been studied before in the framework of the parallel study of language and music. These features are argued to contribute to the effect of ‘expressiveness’ which is here defined as the combination of the acoustic features (variation in duration, pitch, loudness, and articulation) that results in aesthetic appreciation of the linguistic and the musical acoustic stream and which is distinct from pitch, emotional and pragmatic prosody as well as syntactic structure. The present investigation took a neuropsychological approach, comparing the performance of a right temporo-parietal stroke patient IB; a congenitally amusic individual, BZ; and 24 control participants with and without musical training. Apart from the main focus on the perception of ‘expressiveness’, additional aspects of language and music perception were studied. A new battery was designed that consisted of 8 tasks; ‘speech prosody detection’, ‘expressive speech prosody’, ‘expressive music prosody’, ‘emotional speech prosody’, ‘emotional music prosody, ‘speech pitch’, ‘speech rate’, and ‘music tempo’. These tasks addressed both theoretical and methodological issues in this comparative cognitive framework. IB’s performance on the expressive speech prosody task revealed a severe perceptual impairment, whereas his performance on the analogous music task examining ‘expressiveness’ was unimpaired. BZ also performed successfully on the same music task despite being characterised as congenital amusic by an earlier study. Musically untrained controls also had a successful performance. The data from IB suggest that speech and music stimuli encompassing similar features are not necessarily processed by the same mechanisms. These results can have further implications for the approach to the relationship of language and music within the study of cognitive deficits

Investigating the build-up of precedence effect using reflection masking

The auditory processing level involved in the build‐up of precedence [Freyman et al., J. Acoust. Soc. Am. 90, 874–884 (1991)] has been investigated here by employing reflection masked threshold (RMT) techniques. Given that RMT techniques are generally assumed to address lower levels of the auditory signal processing, such an approach represents a bottom‐up approach to the buildup of precedence. Three conditioner configurations measuring a possible buildup of reflection suppression were compared to the baseline RMT for four reflection delays ranging from 2.5–15 ms. No buildup of reflection suppression was observed for any of the conditioner configurations. Buildup of template (decrease in RMT for two of the conditioners), on the other hand, was found to be delay dependent. For five of six listeners, with reflection delay=2.5 and 15 ms, RMT decreased relative to the baseline. For 5‐ and 10‐ms delay, no change in threshold was observed. It is concluded that the low‐level auditory processing involved in RMT is not sufficient to realize a buildup of reflection suppression. This confirms suggestions that higher level processing is involved in PE buildup. The observed enhancement of reflection detection (RMT) may contribute to active suppression at higher processing levels

Paralinguistic vocal control of interactive media: how untapped elements of voice might enhance the role of non-speech voice input in the user's experience of multimedia.

Much interactive media development, especially commercial development, implies the dominance of the visual modality, with sound as a limited supporting channel. The development of multimedia technologies such as augmented reality and virtual reality has further revealed a distinct partiality to visual media. Sound, however, and particularly voice, have many aspects which have yet to be adequately investigated. Exploration of these aspects may show that sound can, in some respects, be superior to graphics in creating immersive and expressive interactive experiences. With this in mind, this thesis investigates the use of non-speech voice characteristics as a complementary input mechanism in controlling multimedia applications. It presents a number of projects that employ the paralinguistic elements of voice as input to interactive media including both screen-based and physical systems. These projects are used as a means of exploring the factors that seem likely to affect users’ preferences and interaction patterns during non-speech voice control. This exploration forms the basis for an examination of potential roles for paralinguistic voice input. The research includes the conceptual and practical development of the projects and a set of evaluative studies. The work submitted for Ph.D. comprises practical projects (50 percent) and a written dissertation (50 percent). The thesis aims to advance understanding of how voice can be used both on its own and in combination with other input mechanisms in controlling multimedia applications. It offers a step forward in the attempts to integrate the paralinguistic components of voice as a complementary input mode to speech input applications in order to create a synergistic combination that might let the strengths of each mode overcome the weaknesses of the other

Cognitive and Neurophysiological Models of Brain Asymmetry

Asymmetry is an inherent characteristic of brain organization in both humans and other vertebrate species, and is evident at the behavioral, neurophysiological, and structural levels. Brain asymmetry underlies the organization of several cognitive systems, such as emotion, communication, and spatial processing. Despite this ubiquity of asymmetries in the vertebrate brain, we are only beginning to understand the complex neuronal mechanisms underlying the interaction between hemispheric asymmetries and cognitive systems. Unfortunately, despite the vast number of empirical studies on brain asymmetries, theoretical models that aim to provide mechanistic explanations of hemispheric asymmetries are sparse in the field. Therefore, this Special Issue aims to highlight empirically based mechanistic models of brain asymmetry. Overall, six theoretical and four empirical articles were published in the Special Issue, covering a wide range of topics, from human handedness to auditory laterality in bats. Two key challenges for theoretical models of brain asymmetry are the integration of increasingly complex molecular data into testable models, and the creation of theoretical models that are robust and testable across different species

Paralinguistic vocal control of interactive media : how untapped elements of voice might enhance the role of non-speech voice input in the user's experience of multimedia

Much interactive media development, especially commercial development, implies the dominance of the visual modality, with sound as a limited supporting channel. The development of multimedia technologies such as augmented reality and virtual reality has further revealed a distinct partiality to visual media. Sound, however, and particularly voice, have many aspects which have yet to be adequately investigated. Exploration of these aspects may show that sound can, in some respects, be superior to graphics in creating immersive and expressive interactive experiences. With this in mind, this thesis investigates the use of non-speech voice characteristics as a complementary input mechanism in controlling multimedia applications. It presents a number of projects that employ the paralinguistic elements of voice as input to interactive media including both screen-based and physical systems. These projects are used as a means of exploring the factors that seem likely to affect users' preferences and interaction patterns during non-speech voice control. This exploration forms the basis for an examination of potential roles for paralinguistic voice input. The research includes the conceptual and practical development of the projects and a set of evaluative studies. The work submitted for Ph.D. comprises practical projects (50 percent) and a written dissertation (50 percent). The thesis aims to advance understanding of how voice can be used both on its own and in combination with other input mechanisms in controlling multimedia applications. It offers a step forward in the attempts to integrate the paralinguistic components of voice as a complementary input mode to speech input applications in order to create a synergistic combination that might let the strengths of each mode overcome the weaknesses of the other.EThOS - Electronic Theses Online ServiceGBUnited Kingdo