Visual attention-capture cue in depicted scenes fails to modulate online sentence processing

Everyday communication is enriched by the visual environment that listeners concomitantly link to the linguistic input. If and when visual cues are integrated into the mental meaning representation of the communicative setting, is still unclear. In our earlier findings, the integration of linguistic cues (i.e., topic-hood of a discourse referent) reduced discourse updating costs of the mental representation as indicated by reduced sentence-initial processing costs of the non-canonical word order in German. In the present study we tried to replicate our earlier findings by replacing the linguistic cue by a visual attention-capture cue presented below the threshold of perception in order to direct participant’s attention to a depicted referent. While this type of cue has previously been shown to modulate word order preferences in sentence production, we found no effects on sentence comprehension. We discuss possible theory-based reasons for the null effect of the implicit visual cue as well as methodological caveats and issues that should be considered in future research on multimodal meaning integration

Linguistic expectation management in online discourse processing: An investigation of Dutch inderdaad 'indeed' and eigenlijk 'actually'

Interpersonal discourse particles (DPs), such as Dutch inderdaad (≈‘indeed’) and eigenlijk (≈‘actually’) are highly frequent in everyday conversational interaction. Despite extensive theoretical descriptions of their polyfunctionality, little is known about how they are used by language comprehenders. In two visual world eye-tracking experiments involving an online dialogue completion task, we asked to what extent inderdaad, confirming an inferred expectation, and eigenlijk, contrasting with an inferred expectation, influence real-time understanding of dialogues. Answers in the dialogues contained a DP or a control adverb, and a critical discourse referent was replaced by a beep; participants chose the most likely dialogue completion by clicking on one of four referents in a display. Results show that listeners make rapid and fine-grained situation-specific inferences about the use of DPs, modulating their expectations about how the dialogue will unfold. Findings further specify and constrain theories about the conversation-managing function and polyfunctionality of DPs

Referential precedents in spoken language comprehension: a review and meta-analysis

Listeners’ interpretations of referring expressions are influenced by referential precedents—temporary conventions established in a discourse that associate linguistic expressions with referents. A number of psycholinguistic studies have investigated how much precedent effects depend on beliefs about the speaker’s perspective versus more egocentric, domain-general processes. We review and provide a meta-analysis of visual-world eyetracking studies of precedent use, focusing on three principal effects: (1) a same speaker advantage for maintained precedents; (2) a different speaker advantage for broken precedents; and (3) an overall main effect of precedents. Despite inconsistent claims in the literature, our combined analysis reveals surprisingly consistent evidence supporting the existence of all three effects, but with different temporal profiles. These findings carry important implications for existing theoretical explanations of precedent use, and challenge explanations based solely on the use of information about speakers’ perspectives

Effects of psychological attention on pronoun comprehension

Pronoun comprehension is facilitated for referents that are focused in the discourse context. Discourse focus has been described as a function of attention, especially shared attention, but few studies have explicitly tested this idea. Two experiments used an exogenous capture cue paradigm to demonstrate that listeners’ visual attention at the onset of a story influences their preferences during pronoun resolution later in the story. In both experiments trial-initial attention modulated listeners’ transitory biases while considering referents for the pronoun, whether it was in response to the capture cue or not. These biases even had a small influence on listeners’ final interpretation of the pronoun. These results provide independently-motivated evidence that the listener’s attention influences the on-line processes of pronoun comprehension. Trial-initial attentional shifts were made on the basis of non-shared, private information, demonstrating that attentional effects on pronoun comprehension are not restricted to shared attention among interlocutors

The effect of executive function on referential use in returnee children

This longitudinal study investigates the choice of referential expressions in English of returnee children. The specific aim of this study is to examine whether there is a correlation between executive functions and choices of referential expressions in returnee children, and how they correlate if there is. Thirty-six Japanese-English speaking returnee bilingual children (returnee children are the group of children from immigrant families who returned to their first language dominant society from their second language dominant society after some time spent there) aged from 7 to 13 participated in a narrative task by telling a story of a wordless book “Frog on his own” (Mayer, 1973) in English and three executive tasks (DCCS, Simon task, and N-back task). There were three testing sessions, administered with a time interval of one year after the first session, and four years after the second one. Children with more advanced executive function skills are predicted to perform bettern the narrative task than other children, namely producing more appropriate referents to avoid ambiguity. Apart from that, crosslinguistic influence is expected during the second and(or) the third session of the narrative task, where the exposure to English decreased drastically while Japanese increased significantly. The results show that mixing cost and L2 exposure affect returnee children’s referential use. That is, children who have smaller mixing costs and sustained English exposure show better referential skills in maintenance by using more pronouns and fewer nominals

Intonation in Language Acquisition - Evidence from German

This dissertation studies the role of intonation in language acquisition. After a general introduction about the phonetic and phonological aspects of intonation and its different forms and functions within language, two different models of language acquisition and the role of intonation within these two models will be presented. Following this, I will present and discuss empirical data on the question, whether young German learning children use intonation in order to acquire language. Two comprehension studies will be presented. Here, I concentrate on the question whether children understand the referential function of intonation and whether they can use this knowledge in order to learn new words. Additionally, I will present empirical evidence that focuses on the question whether children use intonation in resolving participant roles in complex syntactic constructions as well as in resolving syntactic ambiguities development. Finally, I will present two production studies that investigate the prosodic realization of target referents that have different informational statuses within a discourse from both young children and parents, talking to their children. Overall, the data from these studies suggest that language learning children do use the intonational form of an utterance from early on in order to understand another´s intention. Young language learning children do understand that a certain intonational form conveys a function. Additionally, the studies presented in this thesis suggest that children also use intonation in order to convey their own communicative intentions. Thus, intonation is an important instrument for young children‘s language acquisition as they use the information that is provided by intonation, not only to learn words and to combine them to syntactic constructions, but also for the understanding of paralinguistic properties of language. The findings of the studies presented in this thesis are discussed with regard to different theories of language acquisition. Additionally, I will give insight into the understanding of the development of young children´s use of intonation

Consensus Paper: Situated and Embodied Language Acquisition

Theories of embodied cognition postulate that perceptual, sensorimotor, and affective properties of concepts support language learning and processing. In this paper, we argue that language acquisition, as well as processing, is situated in addition to being embodied. In particular, first, it is the situated nature of initial language development that affords for the developing system to become embodied. Second, the situated nature of language use changes across development and adulthood. We provide evidence from empirical studies for embodied effects of perception, action, and valence as they apply to both embodied cognition and situated cognition across developmental stages. Although the evidence is limited, we urge researchers to consider differentiating embodied cognition within situated context, in order to better understand how these separate mechanisms interact for learning to occur. This delineation also provides further clarity to the study of classroom-based applications and the role of embodied and situated cognition in the study of developmental disorders. We argue that theories of language acquisition need to address for the complex situated context of real-world learning by completing a “circular notion”: observing experimental paradigms in real-world settings and taking these observations to later refine lab-based experiments

Interpretation of ambiguous pronouns in adults with intellectual disabilities

Background — Pronouns are referentially ambiguous (e.g. she could refer to any female), yet they are common in everyday conversations. Individuals with typical development (TD) employ several strategies to avoid pronoun interpretation errors, including the subject bias — an assumption that a pronoun typically refers to the subject (or, with the closely related order-of-mention bias, the first-mentioned character) of the previous sentence. However, it is unknown if adults with intellectual disability (ID) share this strategy or the extent to which the subject bias is associated with non-verbal abilities or receptive vocabulary. Methods — We tested 22 adults with mixed-aetiology ID on their interpretation of ambiguous pronouns using the visual world eye-tracking paradigm and by asking a follow-up pronoun interpretation question. A group of TD adults was also tested to establish the strength of the subject bias with our materials and task. Results — Adults with ID did demonstrate the subject bias, but it was significantly less robust than that seen in TD. For participants with ID, the subject bias was influenced by non-verbal IQ and receptive vocabulary at different stages of processing. Conclusions — Given the frequency of pronouns in conversation, strengthening the subject bias may help alleviate discourse and reading comprehension challenges for individuals with ID, particularly those with lower non-verbal and/or vocabulary skills

Rational Redundancy in Situated Communication

Contrary to the Gricean maxims of Quantity (Grice, 1975), it has been repeatedly shown that speakers often include redundant information in their utterances (over- specifications). Previous research on referential communication has long debated whether this redundancy is the result of speaker-internal or addressee-oriented processes, while it is also unclear whether referential redundancy hinders or facilitates comprehension. We present a bounded-rational account of referential redundancy, according to which any word in an utterance, even if it is redundant, can be beneficial to comprehension, to the extent that it facilitates the reduction of listeners’ uncertainty regarding the target referent in a co-present visual scene. Information-theoretic metrics, such as Shannon’s entropy (Shannon, 1948), were employed in order to quantify this uncertainty in bits of information, and gain an estimate of the cognitive effort related to referential processing. Under this account, speakers may, therefore, utilise redundant adjectives in order to reduce the visually-determined entropy (and thereby their listeners’ cognitive effort) more uniformly across their utterances. In a series of experiments, we examined both the comprehension and the production of over-specifications in complex visual contexts. Our findings are in line with the bounded-rational account. Specifically, we present evidence that: (a) in view of complex visual scenes, listeners’ processing and identification of the target referent may be facilitated by the use of redundant adjectives, as well as by a more uniform reduction of uncertainty across the utterance, and (b) that, while both speaker-internal and addressee-oriented processes are at play in the production of over-specifications, listeners’ processing concerns may also influence the encoding of redundant adjectives, at least for some speakers, who encode redundant adjectives more frequently when these adjectives contribute to a more uniform reduction of referential entropy.SFB1102 Information Density and Linguistic Encoding (iDeaL

Individual differences in the production of referential expressions: the effect of language proficiency, language exposure and executive function in bilingual and monolingual children

One hundred and seventy-two English-speaking 5- to 7-year-olds participated in a referential communication task where we manipulated the linguistic mention and the visual presence of a competitor alongside a target referent. Eighty-seven of the children were additionally exposed to a language other than English (bilinguals). We measured children’s language proficiency, verbal working memory (WM), cognitive control skills, family SES, and relative amount of cumulative exposure and use of the home language for the bilinguals. Children’s use of full Noun Phrases (NPs) to identify a target referent was predicted by the visual presence of a competitor more than by its linguistic mention. Verbal WM and proficiency predicted NP use, while cognitive control skills predicted both the ability to use expressions signalling discourse integration and sensitivity to the presence of a discourse competitor, but not of a visual competitor. Bilingual children were as informative as monolingual children once proficiency was controlled for