Speaker Normalization Using Cortical Strip Maps: A Neural Model for Steady State Vowel Identification

Auditory signals of speech are speaker-dependent, but representations of language meaning are speaker-independent. Such a transformation enables speech to be understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitchindependent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by Adaptive Resonance Theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [J. Acoust. Soc. Am. 24, 175-184 (1952)] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Speaker Normalization Using Cortical Strip Maps: A Neural Model for Steady State vowel Categorization

Auditory signals of speech are speaker-dependent, but representations of language meaning are speaker-independent. The transformation from speaker-dependent to speaker-independent language representations enables speech to be learned and understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitch-independent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by Adaptive Resonance Theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [J. Acoust. Soc. Am. 24, 175-184 (1952)] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Recognizing Speech in a Novel Accent: The Motor Theory of Speech Perception Reframed

The motor theory of speech perception holds that we perceive the speech of another in terms of a motor representation of that speech. However, when we have learned to recognize a foreign accent, it seems plausible that recognition of a word rarely involves reconstruction of the speech gestures of the speaker rather than the listener. To better assess the motor theory and this observation, we proceed in three stages. Part 1 places the motor theory of speech perception in a larger framework based on our earlier models of the adaptive formation of mirror neurons for grasping, and for viewing extensions of that mirror system as part of a larger system for neuro-linguistic processing, augmented by the present consideration of recognizing speech in a novel accent. Part 2 then offers a novel computational model of how a listener comes to understand the speech of someone speaking the listener's native language with a foreign accent. The core tenet of the model is that the listener uses hypotheses about the word the speaker is currently uttering to update probabilities linking the sound produced by the speaker to phonemes in the native language repertoire of the listener. This, on average, improves the recognition of later words. This model is neutral regarding the nature of the representations it uses (motor vs. auditory). It serve as a reference point for the discussion in Part 3, which proposes a dual-stream neuro-linguistic architecture to revisits claims for and against the motor theory of speech perception and the relevance of mirror neurons, and extracts some implications for the reframing of the motor theory

Mechanisms of memory retrieval in slow-wave sleep : memory retrieval in slow-wave sleep

Study Objectives: Memories are strengthened during sleep. The benefits of sleep for memory can be enhanced by re-exposing the sleeping brain to auditory cues; a technique known as targeted memory reactivation (TMR). Prior studies have not assessed the nature of the retrieval mechanisms underpinning TMR: the matching process between auditory stimuli encountered during sleep and previously encoded memories. We carried out two experiments to address this issue. Methods: In Experiment 1, participants associated words with verbal and non-verbal auditory stimuli before an overnight interval in which subsets of these stimuli were replayed in slow-wave sleep. We repeated this paradigm in Experiment 2 with the single difference that the gender of the verbal auditory stimuli was switched between learning and sleep. Results: In Experiment 1, forgetting of cued (vs. non-cued) associations was reduced by TMR with verbal and non-verbal cues to similar extents. In Experiment 2, TMR with identical non-verbal cues reduced forgetting of cued (vs. non-cued) associations, replicating Experiment 1. However, TMR with non-identical verbal cues reduced forgetting of both cued and non-cued associations. Conclusions: These experiments suggest that the memory effects of TMR are influenced by the acoustic overlap between stimuli delivered at training and sleep. Our findings hint at the existence of two processing routes for memory retrieval during sleep. Whereas TMR with acoustically identical cues may reactivate individual associations via simple episodic matching, TMR with non-identical verbal cues may utilise linguistic decoding mechanisms, resulting in widespread reactivation across a broad category of memories

Speaker Recognition Systems: A Tutorial

Abstract This paper gives an overview of speaker recognition systems. Speaker recognition is the task of automatically recognizing who is speaking by identifying an unknown speaker among several reference speakers using speaker-specific information included in speech waves. The different classification of speaker recognition and speech processing techniques required for performing the recognition task are discussed. The basic modules of a speaker recognition system are outlined and discussed. Some of the techniques required to implement each module of the system were discussed and others are mentioned. The methods were also compared with one another. Finally, this paper concludes by giving a few research trends in speaker recognition for some year to come

Grounding semantics in robots for Visual Question Answering

In this thesis I describe an operational implementation of an object detection and description system that incorporates in an end-to-end Visual Question Answering system and evaluated it on two visual question answering datasets for compositional language and elementary visual reasoning

Speech Communication

Contains reports on four research projects.C.J. LeBel FellowshipKurzweil Applied IntelligenceNational Institutes of Health (Grant 5 T32 NS07040)National Institutes of Health (Grant 5 RO1 NS04332)National Science Foundation (Grant BNS84-18733)Systems Development FoundationU.S. Navy - Office of Naval Research (Contract N00014-82-K-0727