The Role of Graduality for Referring Expression Generation in Visual Scenes

International audienceReferring Expression Generation (reg) algorithms, a core component of systems that generate text from non-linguistic data, seek to identify domain objects using natural language descriptions. While reg has often been applied to visual domains, very few approaches deal with the problem of fuzziness and gradation. This paper discusses these problems and how they can be accommodated to achieve a more realistic view of the task of referring to objects in visual scenes

The role of graduality for referring expression generation in visual scenes

Referring Expression Generation (reg) algorithms, a core component of systems that generate text from non-linguistic data, seek to identify domain objects using natural language descriptions. While reg has often been applied to visual domains, very few approaches deal with the problem of fuzziness and gradation. This paper discusses these problems and how they can be accommodated to achieve a more realistic view of the task of referring to objects in visual scenes.peer-reviewe

Qualities, objects, sorts, and other treasures : gold digging in English and Arabic

In the present monograph, we will deal with questions of lexical typology in the nominal domain. By the term "lexical typology in the nominal domain", we refer to crosslinguistic regularities in the interaction between (a) those areas of the lexicon whose elements are capable of being used in the construction of "referring phrases" or "terms" and (b) the grammatical patterns in which these elements are involved. In the traditional analyses of a language such as English, such phrases are called "nominal phrases". In the study of the lexical aspects of the relevant domain, however, we will not confine ourselves to the investigation of "nouns" and "pronouns" but intend to take into consideration all those parts of speech which systematically alternate with nouns, either as heads or as modifiers of nominal phrases. In particular, this holds true for adjectives both in English and in other Standard European Languages. It is well known that adjectives are often difficult to distinguish from nouns, or that elements with an overt adjectival marker are used interchangeably with nouns, especially in particular semantic fields such as those denoting MATERIALS or NATlONALlTIES. That is, throughout this work the expression "lexical typology in the nominal domain" should not be interpreted as "a typology of nouns", but, rather, as the cross-linguistic investigation of lexical areas constitutive for "referring phrases" irrespective of how the parts-of-speech system in a specific language is defined

The co-evolution of number concepts and counting words

Humans possess a number concept that differs from its predecessors in animal cognition in two crucial respects: (1) it is based on a numerical sequence whose elements are not confined to quantitative contexts, but can indicate cardinal/quantitative as well as ordinal and even nominal properties of empirical objects (e.g. ‘five buses’: cardinal; ‘the fifth bus’: ordinal; ‘the #5 bus’: nominal), and (2) it can involve recursion and, via recursion, discrete infinity. In contrast to that, the predecessors of numerical cognition that we find in animals and human infants rely on finite and iconic representations that are limited to cardinality and do not support a unified concept of number. In this paper, I argue that the way such a unified number concept could evolve in humans is via verbal sequences that are employed as numerical tools, that is, sequences of words whose elements are associated with empirical objects in number assignments. In particular, I show that a certain kind of number words, namely the counting sequences of natural languages, can be characterised as a central instance of verbal numerical tools. I describe a possible scenario for the emergence of such verbal numerical tools in human history that starts from iconic roots and that suggests that in a process of co-evolution, the gradual emergence of counting sequences and the development of an increasingly comprehensive number concept supported each other. On this account, it is language that opened the way for numerical cognition, suggesting that it is no accident that the same species that possesses the language faculty as a unique trait, should also be the one that developed a systematic concept of number

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Anatomy and the type concept in biology show that ontologies must be adapted to the diagnostic needs of research

Background: In times of exponential data growth in the life sciences, machine-supported approaches are becoming increasingly important and with them the need for FAIR (Findable, Accessible, Interoperable, Reusable) and eScience-compliant data and metadata standards. Ontologies, with their queryable knowledge resources, play an essential role in providing these standards. Unfortunately, biomedical ontologies only provide ontological definitions that answer What is it? questions, but no method-dependent empirical recognition criteria that answer How does it look? questions. Consequently, biomedical ontologies contain knowledge of the underlying ontological nature of structural kinds, but often lack sufficient diagnostic knowledge to unambiguously determine the reference of a term. Results: We argue that this is because ontology terms are usually textually defined and conceived as essentialistic classes, while recognition criteria often require perception-based definitions because perception-based contents more efficiently document and communicate spatial and temporal information—a picture is worth a thousand words. Therefore, diagnostic knowledge often must be conceived as cluster classes or fuzzy sets. Using several examples from anatomy, we point out the importance of diagnostic knowledge in anatomical research and discuss the role of cluster classes and fuzzy sets as concepts of grouping needed in anatomy ontologies in addition to essentialistic classes. In this context, we evaluate the role of the biological type concept and discuss its function as a general container concept for groupings not covered by the essentialistic class concept. Conclusions: We conclude that many recognition criteria can be conceptualized as text-based cluster classes that use terms that are in turn based on perception-based fuzzy set concepts. Finally, we point out that only if biomedical ontologies model also relevant diagnostic knowledge in addition to ontological knowledge, they will fully realize their potential and contribute even more substantially to the establishment of FAIR and eScience-compliant data and metadata standards in the life sciences

Anatomy and the type concept in biology show that ontologies must be adapted to the diagnostic needs of research

Background: In times of exponential data growth in the life sciences, machine-supported approaches are becoming increasingly important and with them the need for FAIR (Findable, Accessible, Interoperable, Reusable) and eScience-compliant data and metadata standards. Ontologies, with their queryable knowledge resources, play an essential role in providing these standards. Unfortunately, biomedical ontologies only provide ontological definitions that answer What is it? questions, but no method-dependent empirical recognition criteria that answer How does it look? questions. Consequently, biomedical ontologies contain knowledge of the underlying ontological nature of structural kinds, but often lack sufficient diagnostic knowledge to unambiguously determine the reference of a term. Results: We argue that this is because ontology terms are usually textually defined and conceived as essentialistic classes, while recognition criteria often require perception-based definitions because perception-based contents more efficiently document and communicate spatial and temporal information—a picture is worth a thousand words. Therefore, diagnostic knowledge often must be conceived as cluster classes or fuzzy sets. Using several examples from anatomy, we point out the importance of diagnostic knowledge in anatomical research and discuss the role of cluster classes and fuzzy sets as concepts of grouping needed in anatomy ontologies in addition to essentialistic classes. In this context, we evaluate the role of the biological type concept and discuss its function as a general container concept for groupings not covered by the essentialistic class concept. Conclusions: We conclude that many recognition criteria can be conceptualized as text-based cluster classes that use terms that are in turn based on perception-based fuzzy set concepts. Finally, we point out that only if biomedical ontologies model also relevant diagnostic knowledge in addition to ontological knowledge, they will fully realize their potential and contribute even more substantially to the establishment of FAIR and eScience-compliant data and metadata standards in the life sciences

Referring Expression Generation in Situated Interaction

While most current frameworks for reference handling are based on binary truth-theoretic knowledge representation, in this thesis I argue for a perspective on reference which emphasises the collaborative nature of reference. I present the Probabilistic Reference And GRounding mechanism (PRAGR) which uses flexible concept assignment based on vague property models and situational context in order to maximise the chance of communicative success. I demonstrate that PRAGR is capable of dealing with several property domains with different internal structures, such as graded adjectives, colour, shape, projective terms, and projective regions. Further, I show that PRAGR is fit to handle in an integrated fashion the most relevant challenges of Referring Expression Generation, in particular graded properties, spatial relations, and salience effects. In three empirical evaluation studies, I demonstrate the usefulness of PRAGR for situated referential communication