Collaboration on reference to objects that are not mutually known

In conversation, a person sometimes has to refer to an object that is not previously known to the other participant. We present a plan-based model of how agents collaborate on reference of this sort. In making a reference, an agent uses the most salient attributes of the referent. In understanding a reference, an agent determines his confidence in its adequacy as a means of identifying the referent. To collaborate, the agents use judgment, suggestion, and elaboration moves to refashion an inadequate referring expression.Comment: 6 pages, to appear in proceedings of COLING-94, LaTeX (now uses fullname.sty, fullname.bst

The Role of Graduality for Referring Expression Generation in Visual Scenes

International audienceReferring Expression Generation (reg) algorithms, a core component of systems that generate text from non-linguistic data, seek to identify domain objects using natural language descriptions. While reg has often been applied to visual domains, very few approaches deal with the problem of fuzziness and gradation. This paper discusses these problems and how they can be accommodated to achieve a more realistic view of the task of referring to objects in visual scenes

The role of graduality for referring expression generation in visual scenes

Referring Expression Generation (reg) algorithms, a core component of systems that generate text from non-linguistic data, seek to identify domain objects using natural language descriptions. While reg has often been applied to visual domains, very few approaches deal with the problem of fuzziness and gradation. This paper discusses these problems and how they can be accommodated to achieve a more realistic view of the task of referring to objects in visual scenes.peer-reviewe

A closer look at referring expressions for video object segmentation

The task of Language-guided Video Object Segmentation (LVOS) aims at generating binary masks for an object referred by a linguistic expression. When this expression unambiguously describes an object in the scene, it is named referring expression (RE). Our work argues that existing benchmarks used for LVOS are mainly composed of trivial cases, in which referents can be identified with simple phrases. Our analysis relies on a new categorization of the referring expressions in the DAVIS-2017 and Actor-Action datasets into trivial and non-trivial REs, where the non-trivial REs are further annotated with seven RE semantic categories. We leverage these data to analyze the performance of RefVOS, a novel neural network that obtains competitive results for the task of language-guided image segmentation and state of the art results for LVOS. Our study indicates that the major challenges for the task are related to understanding motion and static actions.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was partially supported by the projects PID2019-107255GB-C22 and PID2020-117142GB-I00 funded by MCIN/ AEI /10.13039/501100011033 Spanish Ministry of Science, and the grant 2017-SGR-1414 of the Government of Catalonia. This work was also partially supported by the project RTI2018-095232-B-C22 funded by the Spanish Ministry of Science, Innovation and Universities.Peer ReviewedPostprint (published version

Lexical choice for complex noun phrases: Structure, modifiers, and determiners

This paper presents a lexical choice component for complex noun phrases. We first explain why lexical choice for NPs deserves special attention within the standard pipeline architecture for a generator. The task of the lexical chooser for NPs is more complex than for clauses because the syntax of NPs is less understood than for clauses, and therefore, syntactic realization components, while they accept a predicate-argument structure as input for clauses, require a purely syntactic tree as input for NPs. The task of mapping conceptual relations to different syntactic modifiers is therefore left to the lexical chooser for NPs. The paper focuses on the syntagmatic aspect of lexical choice, identifying a process called “NP planning”. It focuses on a set of communicative goals that NPs can satisfy and specifies an interface between the different components of the generator and the lexical chooser. The technique presented for NP planning encapsulates a rich lexical knowledge and allows for the generation of a wide variety of syntactic constructions. It also allows for a large paraphrasing power because it dynamically maps conceptual information to various syntactic slots

Integrating Referring and Informing in NP Planning

Two of the functions of an NP are to refer (identify a particular entity) and to inform (provide new information about an entity). While many NPs may serve only one of these functions, some NPs conflate the functions, not only referring but also providing new information about the referent. For instance, this delicious apple indicates not only which apple the speaker is referring to, but also provides information as to the speaker's appreciation of the apple. This paper describes an implemented NPplanning system which integrates informing into the referring expression generation process. The integration involves allowing informing to influence decisions at each stage of the formation of the referring form, including: the selection of the form of the NP; the choice of the head of a common NP; the choice of the Deictic in common NPs; the choice of restrictive modifiers, and the inclusion of non-referring modifiers. The system is domain-independent, and is presently functioning within a full text generation system

Langue, dialogue finalisé et cognition spatiale

L'objectif de ce chapitre est d'aborder l'espace sous l'angle de la communication homme-machine pour mettre en évidence ce qui peut caractériser les mécanismes de compréhension d'expression de l'espace, ainsi que les opérations sous-jacentes de perception et de raisonnement. Nous montrons comment la perspective de réaliser des systèmes opérationnels de dialogue homme-machine permet d'intégrer la modélisation de l'espace dans une perspective plus large intégrant de nombreux paramètres liés à tout acte de communication