Time and information in perceptual adaptation to speech

Presubmission manuscript and supplementary files (stimuli, stimulus presentation code, data, data analysis code).Perceptual adaptation to a talker enables listeners to efficiently resolve the many-to-many mapping between variable speech acoustics and abstract linguistic representations. However, models of speech perception have not delved into the variety or the quantity of information necessary for successful adaptation, nor how adaptation unfolds over time. In three experiments using speeded classification of spoken words, we explored how the quantity (duration), quality (phonetic detail), and temporal continuity of talker-specific context contribute to facilitating perceptual adaptation to speech. In single- and mixed-talker conditions, listeners identified phonetically-confusable target words in isolation or preceded by carrier phrases of varying lengths and phonetic content, spoken by the same talker as the target word. Word identification was always slower in mixed-talker conditions than single-talker ones. However, interference from talker variability decreased as the duration of preceding speech increased but was not affected by the amount of preceding talker-specific phonetic information. Furthermore, efficiency gains from adaptation depended on temporal continuity between preceding speech and the target word. These results suggest that perceptual adaptation to speech may be understood via models of auditory streaming, where perceptual continuity of an auditory object (e.g., a talker) facilitates allocation of attentional resources, resulting in more efficient perceptual processing.NIH NIDCD (R03DC014045

Computational and Robotic Models of Early Language Development: A Review

We review computational and robotics models of early language learning and development. We first explain why and how these models are used to understand better how children learn language. We argue that they provide concrete theories of language learning as a complex dynamic system, complementing traditional methods in psychology and linguistics. We review different modeling formalisms, grounded in techniques from machine learning and artificial intelligence such as Bayesian and neural network approaches. We then discuss their role in understanding several key mechanisms of language development: cross-situational statistical learning, embodiment, situated social interaction, intrinsically motivated learning, and cultural evolution. We conclude by discussing future challenges for research, including modeling of large-scale empirical data about language acquisition in real-world environments. Keywords: Early language learning, Computational and robotic models, machine learning, development, embodiment, social interaction, intrinsic motivation, self-organization, dynamical systems, complexity.Comment: to appear in International Handbook on Language Development, ed. J. Horst and J. von Koss Torkildsen, Routledg

Perceptual-gestural (mis)mapping in serial short-term memory: The impact of talker variability

The mechanisms underlying the poorer serial recall of talker-variable lists (e.g., alternating female–male voices) as compared with single-voice lists were examined. We tested the novel hypothesis that this talker variability effect arises from the tendency for perceptual organization to partition the list into streams based on voice such that the representation of order maps poorly onto the formation of a gestural sequence-output plan assembled in support of the reproduction of the true temporal order of the items. In line with the hypothesis, (a) the presence of a spoken lead-in designed to further promote by-voice perceptual partitioning accentuates the effect (Experiments 1 and 2); (b) the impairment is larger the greater the acoustic coherence is between nonadjacent items: Alternating-voice lists are more poorly recalled than four-voice lists (Experiment 3); and (c) talker variability combines nonadditively with phonological similarity, consistent with the view that both variables disrupt sequence output planning (Experiment 4). The results support the view that serial short-term memory performance reflects the action of sequencing processes embodied within general-purpose perceptual input-processing and gestural output-planning systems

Towards an Indexical Model of Situated Language Comprehension for Cognitive Agents in Physical Worlds

We propose a computational model of situated language comprehension based on the Indexical Hypothesis that generates meaning representations by translating amodal linguistic symbols to modal representations of beliefs, knowledge, and experience external to the linguistic system. This Indexical Model incorporates multiple information sources, including perceptions, domain knowledge, and short-term and long-term experiences during comprehension. We show that exploiting diverse information sources can alleviate ambiguities that arise from contextual use of underspecific referring expressions and unexpressed argument alternations of verbs. The model is being used to support linguistic interactions in Rosie, an agent implemented in Soar that learns from instruction.Comment: Advances in Cognitive Systems 3 (2014

Effects of corrective feedback on EFL speaking task complexity in China’s university classroom

Corrective feedback (CF) and task complexity are two important pedagogical topics in second language acquisition research in recent years, but there is few research investigating effects of CF on speaking task complexity in China’s university classroom settings. This research, through conducting different versions of speaking task experiments among 24 university students in China, explores the effect of teachers’ CF on English as a Foreign Language (EFL) speaking task complexity. According to the analysis of first-hand data, this research finds CF has different effects on EFL oral production with different task complexity. In simple speaking task, the effects of five kinds of CF (from largest to smallest) are listed as follows: clarification quest, metalinguistic feedback, recast, repetition and confirmation check. Regarding complex speaking task, the effects of five categorized CF are ranked from largest to smallest as follows: metalinguistic feedback, confirmation check, recast, clarification request and repetition. Improving to provide CF in pedagogical practice is an important contribution to promote EFL speaking task, so, on the basis of above research results, appropriate ways and forms of providing CF are expected to promote efficiency of CF in EFL classroom under the context of Chinese university classroom

Conflict monitoring in speech processing: an fMRI study of error detection in speech production and perception

To minimize the number of errors in speech, and thereby facilitate communication, speech is monitored before articulation. It is, however, unclear at which level during speech production monitoring takes place, and what mechanisms are used to detect and correct errors. The present study investigated whether internal verbal monitoring takes place through the speech perception system, as proposed by perception-based theories of speech monitoring, or whether mechanisms independent of perception are applied, as proposed by production-based theories of speech monitoring. With the use of fMRI during a tongue twister task we observed that error detection in internal speech during noise-masked overt speech production and error detection in speech perception both recruit the same neural network, which includes pre-supplementary motor area (pre-SMA), dorsal anterior cingulate cortex (dACC), anterior insula (AI), and inferior frontal gyrus (IFG). Although production and perception recruit similar areas, as proposed by perception-based accounts, we did not find activation in superior temporal areas (which are typically associated with speech perception) during internal speech monitoring in speech production as hypothesized by these accounts. On the contrary, results are highly compatible with a domain general approach to speech monitoring, by which internal speech monitoring takes place through detection of conflict between response options, which is subsequently resolved by a domain general executive center (e.g., the ACC)

Amodal Atypical Neural Oscillatory Activity in Dyslexia: A Cross-Linguistic Perspective

First Published December 21, 2016It has been proposed that atypical neural oscillations in both the auditory and the visual modalities could explain why some individuals fail to learn to read and suffer from developmental dyslexia. However, the role of specific oscillatory mechanisms in reading acquisition is still under debate. In this article, we take a cross-linguistic approach and argue that both the phonological and orthographic specifics of a language (e.g., linguistic rhythm, orthographic depth) shape the oscillatory activity thought to contribute to reading development. The proposed theoretical framework should allow future research to test cross-linguistic hypotheses that will shed light on the heterogeneity of auditory and visual disorders and their underlying brain dysfunction(s) in developmental dyslexia, and inform clinical practice by helping us to diagnose dyslexia across languages.This research was funded by the European Research Council (ERC Advanced Grant, BILITERACY Project, to M.C.), and the Spanish government (Plan Nacional-PSI2012-32128 and PSI2015-65338-P to M.L., Plan Nacional-PSI2012-32350 and PSI2015-65694-P to N.M., and Plan Nacional-PSI2015-67353-R to M.C.). The Basque Center on Brain Cognition and Language acknowledges funding from Ayuda Centro de Excelencia Severo Ochoa SEV-2015-0490