The positive side of a negative reference: the delay between linguistic processing and common ground

Interlocutors converge on names to refer to entities. For example, a speaker might refer to a novel looking object as the jellyfish and, once identified, the listener will too. The hypothesized mechanism behind such referential precedents is a subject of debate. The common ground view claims that listeners register the object as well as the identity of the speaker who coined the label. The linguistic view claims that, once established, precedents are treated by listeners like any other linguistic unit, i.e. without needing to keep track of the speaker. To test predictions from each account, we used visual-world eyetracking, which allows observations in real time, during a standard referential communication task. Participants had to select objects based on instructions from two speakers. In the critical condition, listeners sought an object with a negative reference such as not the jellyfish. We aimed to determine the extent to which listeners rely on the linguistic input, common ground or both. We found that initial interpretations were based on linguistic processing only and that common ground considerations do emerge but only after 1000 ms. Our findings support the idea that-at least temporally-linguistic processing can be isolated from common ground

Between anaphora and deixis...the resolution of the demonstrative noun-phrase ‘that N’

Three experiments examined the hypothesis that the demonstrative noun phrase (NP) that N, as an anadeictic expression, preferentially refers to the less salient referent in a discourse representation when used anaphorically, whereas the anaphoric pronoun he or she preferentially refers to the highly-focused referent. The findings, from a sentence completion task and two reading time experiments that used gender to create ambiguous and unambiguous coreference, reveal that the demonstrative NP specifically orients processing toward a less salient referent when there is no gender cue discriminating between different possible referents. These findings show the importance of taking into account the discourse function of the anaphor itself and its influence on the process of searching for the referent

Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queries

Recognising objects according to a pre-defined fixed set of class labels has been well studied in the Computer Vision. There are a great many practical applications where the subjects that may be of interest are not known beforehand, or so easily delineated, however. In many of these cases natural language dialog is a natural way to specify the subject of interest, and the task achieving this capability (a.k.a, Referring Expression Comprehension) has recently attracted attention. To this end we propose a unified framework, the ParalleL AttentioN (PLAN) network, to discover the object in an image that is being referred to in variable length natural expression descriptions, from short phrases query to long multi-round dialogs. The PLAN network has two attention mechanisms that relate parts of the expressions to both the global visual content and also directly to object candidates. Furthermore, the attention mechanisms are recurrent, making the referring process visualizable and explainable. The attended information from these dual sources are combined to reason about the referred object. These two attention mechanisms can be trained in parallel and we find the combined system outperforms the state-of-art on several benchmarked datasets with different length language input, such as RefCOCO, RefCOCO+ and GuessWhat?!.Comment: 11 page