Do You See What I Mean? Visual Resolution of Linguistic Ambiguities

Understanding language goes hand in hand with the ability to integrate complex contextual information obtained via perception. In this work, we present a novel task for grounded language understanding: disambiguating a sentence given a visual scene which depicts one of the possible interpretations of that sentence. To this end, we introduce a new multimodal corpus containing ambiguous sentences, representing a wide range of syntactic, semantic and discourse ambiguities, coupled with videos that visualize the different interpretations for each sentence. We address this task by extending a vision model which determines if a sentence is depicted by a video. We demonstrate how such a model can be adjusted to recognize different interpretations of the same underlying sentence, allowing to disambiguate sentences in a unified fashion across the different ambiguity types.Comment: EMNLP 201

Teaching learners to communicate effectively in the L2: Integrating body language in the students\u2019 syllabus

In communication a great deal of meaning is exchanged through body language, including gaze, posture, hand gestures and body movements. Body language is largely culture-specific, and rests, for its comprehension, on people\u2019s sharing socio-cultural and linguistic norms. In cross-cultural communication, L2 speakers\u2019 use of body language may convey meaning that is not understood or misinterpreted by the interlocutors, affecting the pragmatics of communication. In spite of its importance for cross-cultural communication, body language is neglected in ESL/EFL teaching. This paper argues that the study of body language should be integrated in the syllabus of ESL/EFL teaching and learning. This is done by: 1) reviewing literature showing the tight connection between language, speech and gestures and the problems that might arise in cross-cultural communication when speakers use and interpret body language according to different conventions; 2) reporting the data from two pilot studies showing that L2 learners transfer L1 gestures to the L2 and that these are not understood by native L2 speakers; 3) reporting an experience teaching body language in an ESL/EFL classroom. The paper suggests that in multicultural ESL/EFL classes teaching body language should be aimed primarily at raising the students\u2019 awareness of the differences existing across cultures

Spontaneous eye movements during passive spoken language comprehension reflect grammatical processing

Language is tightly connected to sensory and motor systems. Recent research using eye- tracking typically relies on constrained visual contexts, viewing a small array of objects on a computer screen. Some critiques of embodiment ask if people simply match their simulations to the pictures being presented. This study compared the comprehension of verbs with two different grammatical forms: the past progressive form (e.g., was walking), which emphasizes the ongoing nature of actions, and the simple past (e.g., walked), which emphasizes the end-state of an action. The results showed that the distribution and timing of eye movements mirrors the underlying conceptual structure of this linguistic difference in the absence of any visual stimuli. Thus, eye movement data suggest that visual inputs are unnecessary to solicit perceptual simulations

Institutionalizing an environmental agenda in business

This paper studies the extent and manner in which an environmental agenda is being institutionalised in two companies in the UK water industry through discursive practices. The paper draws mostly on Fairclough's understanding of critical discourse analysis, taking into account the three dimensions of text, discourse practice, and socio-cultural practice. Analyzing managerial discourse - mostly in the form of interviews - over a period of five years, the paper looks at the discursive means by which a pro-environmental agenda is being established, legitimized and supported. The analysis also pays special attention to the degree of homogeneity or contestedness of environmental discourse within and between the two companies. It concludes that a pro-environmental agenda has been established to some extent in both companies, legitmized mostly through reference to competitive advantage and the law. However, there are differences in the way in which this agenda is legitimized and supported and in the degree to which the company's emvironmental strategy is contested internally. Some conclusions are drawn regarding the likely way in which companies will establish and support environmental concerns and the implications this may have for a wider environmental agenda and public policy

Examining the cognitive costs of counterfactual language comprehension: Evidence from ERPs

Recent empirical research suggests that understanding a counterfactual event (e.g. ‘If Josie had revised, she would have passed her exams’) activates mental representations of both the factual and counterfactual versions of events. However, it remains unclear when readers switch between these models during comprehension, and whether representing multiple ‘worlds’ is cognitively effortful. This paper reports two ERP studies where participants read contexts that set up a factual or counterfactual scenario, followed by a second sentence describing a consequence of this event. Critically, this sentence included a noun that was either consistent or inconsistent with the preceding context, and either included a modal verb to indicate reference to the counterfactual-world or not (thus referring to the factual-world). Experiment 2 used adapted versions of the materials used in Experiment 1 to examine the degree to which representing multiple versions of a counterfactual situation makes heavy demands on cognitive resources by measuring individuals’ working memory capacity. Results showed that when reference to the counterfactual-world was maintained by the ongoing discourse, readers correctly interpreted events according to the counterfactual-world (i.e. showed larger N400 for inconsistent than consistent words). In contrast, when cues referred back to the factual-world, readers showed no difference between consistent and inconsistent critical words, suggesting that they simultaneously compared information against both possible worlds. These results support previous dual-representation accounts for counterfactuals, and provide new evidence that linguistic cues can guide the reader in selecting which world model to evaluate incoming information against. Crucially, we reveal evidence that maintaining and updating a hypothetical model over time relies upon the availability of cognitive resources