More cat than cute? Interpretable Prediction of Adjective-Noun Pairs

The increasing availability of affect-rich multimedia resources has bolstered interest in understanding sentiment and emotions in and from visual content. Adjective-noun pairs (ANP) are a popular mid-level semantic construct for capturing affect via visually detectable concepts such as "cute dog" or "beautiful landscape". Current state-of-the-art methods approach ANP prediction by considering each of these compound concepts as individual tokens, ignoring the underlying relationships in ANPs. This work aims at disentangling the contributions of the `adjectives' and `nouns' in the visual prediction of ANPs. Two specialised classifiers, one trained for detecting adjectives and another for nouns, are fused to predict 553 different ANPs. The resulting ANP prediction model is more interpretable as it allows us to study contributions of the adjective and noun components. Source code and models are available at https://imatge-upc.github.io/affective-2017-musa2/ .Comment: Oral paper at ACM Multimedia 2017 Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes (MUSA2

Beyond the realm of noun and verb: the cognitive lexicon of the young child

Most studies of early lexical development are focused on the acquisition of the noun or verb categories. Only studies targeting the very beginning of word production describe the rich pattern of reference and expressive words produced by very young children. Still, during their second year, children’s production in tokens contains as many words that are not nouns and verbs than words that are. The importance of categories such as communicators, adverbs, pointers and adjectives never decreases, neither in English nor in French children between the age of 1;6 to 2;6. A cross-linguistic comparison shows that the same type of words is the most frequent in English and French children, while a comparison with adult production shows that, in neither language, do the words produced by children match exactly the words they hear most frequently. The difference in the syntactic structure of English and French argues strongly for a cognitive origin to this close match of the children’s words. These words other than nouns and verbs are more complex than they appear, because they cover a whole range of reference principles – direct reference, indirect reference, shared reference, generic reference, multiple reference, ambiguity, similarity, repetition, absence of –, as well as a wide range of expressive meanings. This type of words appears and grows throughout the children’s second year and provides the basic stones for further lexicon and syntax development

Lexical Similarities and Differences in the Mathematics, Science and English Language Textbooks

The teaching of Science and Math in English in Malaysia is an area of great concern to educators and students alike. This study looks, in particular, at the common word classes among keywords identified in the Science, Math and English language Form One textbooks used in Malaysia and the differences in language use identified in the Science and Math textbooks

Disambiguating Nouns, Verbs, and Adjectives Using Automatically Acquired Selectional Preferences

Selectional preferences have been used by word sense disambiguation (WSD) systems as one source of disambiguating information. We evaluate WSD using selectional preferences acquired for English adjective—noun, subject, and direct object grammatical relationships with respect to a standard test corpus. The selectional preferences are specific to verb or adjective classes, rather than individual word forms, so they can be used to disambiguate the co-occurring adjectives and verbs, rather than just the nominal argument heads. We also investigate use of the one-senseper-discourse heuristic to propagate a sense tag for a word to other occurrences of the same word within the current document in order to increase coverage. Although the preferences perform well in comparison with other unsupervised WSD systems on the same corpus, the results show that for many applications, further knowledge sources would be required to achieve an adequate level of accuracy and coverage. In addition to quantifying performance, we analyze the results to investigate the situations in which the selectional preferences achieve the best precision and in which the one-sense-per-discourse heuristic increases performance

AudioPairBank: Towards A Large-Scale Tag-Pair-Based Audio Content Analysis

Recently, sound recognition has been used to identify sounds, such as car and river. However, sounds have nuances that may be better described by adjective-noun pairs such as slow car, and verb-noun pairs such as flying insects, which are under explored. Therefore, in this work we investigate the relation between audio content and both adjective-noun pairs and verb-noun pairs. Due to the lack of datasets with these kinds of annotations, we collected and processed the AudioPairBank corpus consisting of a combined total of 1,123 pairs and over 33,000 audio files. One contribution is the previously unavailable documentation of the challenges and implications of collecting audio recordings with these type of labels. A second contribution is to show the degree of correlation between the audio content and the labels through sound recognition experiments, which yielded results of 70% accuracy, hence also providing a performance benchmark. The results and study in this paper encourage further exploration of the nuances in audio and are meant to complement similar research performed on images and text in multimedia analysis.Comment: This paper is a revised version of "AudioSentibank: Large-scale Semantic Ontology of Acoustic Concepts for Audio Content Analysis

Acquisition of gender agreement in Lithuanian:exploring the effect of diminutive usage in an elicited production task

This study examines Lithuanian children's acquisition of gender agreement using an elicited production task. Lithuanian is a richly inflected Baltic language, with two genders and seven cases. Younger (N=24, mean 3;1, 2;5–3;8) and older (N=24, mean 6;3, 5;6–6;9) children were shown pictures of animals and asked to describe them after hearing the animal's name. Animal names differed with respect to familiarity (novel vs. familiar), derivational status (diminutive vs. simplex) and gender (masculine vs. feminine). Analyses of gender-agreement errors based on adjective and pronoun usage indicated that younger children made more errors than older children, with errors more prevalent for novel animal names. For novel animals, and for feminine nouns, children produced fewer errors with nouns introduced in diminutive form. These results complement findings from several Slavic languages (Russian, Serbian and Polish) that diminutives constitute a salient cluster of word forms that may provide an entry point for the child's acquisition of noun morphology

Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Tasks

An important goal of computer vision is to build systems that learn visual representations over time that can be applied to many tasks. In this paper, we investigate a vision-language embedding as a core representation and show that it leads to better cross-task transfer than standard multi-task learning. In particular, the task of visual recognition is aligned to the task of visual question answering by forcing each to use the same word-region embeddings. We show this leads to greater inductive transfer from recognition to VQA than standard multitask learning. Visual recognition also improves, especially for categories that have relatively few recognition training labels but appear often in the VQA setting. Thus, our paper takes a small step towards creating more general vision systems by showing the benefit of interpretable, flexible, and trainable core representations.Comment: Accepted in ICCV 2017. The arxiv version has an extra analysis on correlation with human attentio