An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal

Investigating the timecourse of accessing conversational implicatures during incremental sentence interpretation

Many contextual inferences in utterance interpretation are explained as following from the nature of conversation and the assumption that participants are rational. Recent psycholinguistic research has focussed on certain of these ‘Gricean’ inferences and have revealed that comprehenders can access them in online interpretation. However there have been mixed results as to the time-course of access. Some results show that Gricean inferences can be accessed very rapidly, as rapidly as any other contextually specified information (Sedivy, 2003; Grodner, Klein, Carbery, & Tanenhaus, 2010); while other studies looking at the same kind of inference suggest that access to Gricean inferences are delayed relative to other aspects of semantic interpretation (Huang & Snedeker, 2009; in press). While previous timecourse research has focussed on Gricean inferences that support the online assignment of reference to definite expressions, the study reported here examines the timecourse of access to scalar implicatures, which enrich the meaning of an utterance beyond the semantic interpretation. Even if access to Gricean inference in support of reference assignment may be rapid, it is still unknown whether genuinely enriching scalar implicatures are delayed. Our results indicate that scalar implicatures are accessed as rapidly as other contextual inferences. The implications of our results are discussed in reference to the architecture of language comprehension

Towards a complete multiple-mechanism account of predictive language processing [Commentary on Pickering & Garrod]

Although we agree with Pickering & Garrod (P&G) that prediction-by-simulation and prediction-by-association are important mechanisms of anticipatory language processing, this commentary suggests that they: (1) overlook other potential mechanisms that might underlie prediction in language processing, (2) overestimate the importance of prediction-by-association in early childhood, and (3) underestimate the complexity and significance of several factors that might mediate prediction during language processing

Self-, other-, and joint monitoring using forward models

In the psychology of language, most accounts of self-monitoring assume that it is based on comprehension. Here we outline and develop the alternative account proposed by Pickering and Garrod (2013), in which speakers construct forward models of their upcoming utterances and compare them with the utterance as they produce them. We propose that speakers compute inverse models derived from the discrepancy (error) between the utterance and the predicted utterance and use that to modify their production command or (occasionally) begin anew. We then propose that comprehenders monitor other people’s speech by simulating their utterances using covert imitation and forward models, and then comparing those forward models with what they hear. They use the discrepancy to compute inverse models and modify their representation of the speaker’s production command, or realize that their representation is incorrect and may develop a new production command. We then discuss monitoring in dialogue, paying attention to sequential contributions, concurrent feedback, and the relationship between monitoring and alignment

Speaker Normalization Using Cortical Strip Maps: A Neural Model for Steady State vowel Categorization

Auditory signals of speech are speaker-dependent, but representations of language meaning are speaker-independent. The transformation from speaker-dependent to speaker-independent language representations enables speech to be learned and understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitch-independent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by Adaptive Resonance Theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [J. Acoust. Soc. Am. 24, 175-184 (1952)] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Affective Music Information Retrieval

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called \emph{acoustic emotion Gaussians} (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a larger-scale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.Comment: 40 pages, 18 figures, 5 tables, author versio

Tentative Reference Acts? ‘Recognitional Demonstratives’ as Means of Suggesting Mutual Knowledge – or Overriding a Lack of It

In an explorative study on German oral corpus data we investigate recognitional use of proximal demonstratives as a means of explicit speaker-hearer interaction shaping the discourse structure. We show that recognitionals mark tentative reference acts in that speakers suggest - or pretend - mutual knowledge of the referent, at the same time appealing to the hearers to accept the reference. Hearers may tacitly or explicitly accept the referential act or deny it asking for clarification, in the latter case making speakers change the intended local discourse topic. On these grounds we argue against a differentiation between recognitional and indefinite demonstratives, subsuming both as kinds of recognitional use under ‘pretended’ cognitive proximity

Lexical Flexibility, Natural Language, and Ontology

The Realist that investigates questions of ontology by appeal to the quantificational structure of language assumes that the semantics for the privileged language of ontology is externalist. I argue that such a language cannot be (some variant of) a natural language, as some Realists propose. The flexibility exhibited by natural language expressions noted by Chomsky and others cannot obviously be characterized by the rigid models available to the externalist. If natural languages are hostile to externalist treatments, then the meanings of natural language expressions serve as poor guides for ontological investigation, insofar as their meanings will fail to determine the referents of their constituents. This undermines the Realist’s use of natural languages to settle disputes in metaphysics

Reclamation: Taking Back Control of Words

Reclamation is the phenomenon of an oppressed group repurposing language to its own ends. A case study is reclamation of slur words. Popa-Wyatt and Wyatt (2018) argued that a slurring utterance is a speech act which performs a discourse role assignment. It assigns a subordinate role to the target, while the speaker assumes a dominant role. This pair of role assignments is used to oppress the target. Here I focus on how reclamation works and under what conditions its benefits can stabilise. I start by reviewing the data and describing preconditions and motivations for reclamation. Can reclamation be explained in the same basic framework as regular slurring utterances? I argue that it can. I also identify some features that must be a prediction of any theory of reclamation. I conclude that reclamation is an instance of a much broader class of acts we do with words to change the distribution of power: it begets power, but it also requires it