Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection

Multi-label image classification is a fundamental but challenging task towards general visual understanding. Existing methods found the region-level cues (e.g., features from RoIs) can facilitate multi-label classification. Nevertheless, such methods usually require laborious object-level annotations (i.e., object labels and bounding boxes) for effective learning of the object-level visual features. In this paper, we propose a novel and efficient deep framework to boost multi-label classification by distilling knowledge from weakly-supervised detection task without bounding box annotations. Specifically, given the image-level annotations, (1) we first develop a weakly-supervised detection (WSD) model, and then (2) construct an end-to-end multi-label image classification framework augmented by a knowledge distillation module that guides the classification model by the WSD model according to the class-level predictions for the whole image and the object-level visual features for object RoIs. The WSD model is the teacher model and the classification model is the student model. After this cross-task knowledge distillation, the performance of the classification model is significantly improved and the efficiency is maintained since the WSD model can be safely discarded in the test phase. Extensive experiments on two large-scale datasets (MS-COCO and NUS-WIDE) show that our framework achieves superior performances over the state-of-the-art methods on both performance and efficiency.Comment: accepted by ACM Multimedia 2018, 9 pages, 4 figures, 5 table

A Semantic neighborhood approach to relatedness evaluation on well-founded domain ontologies

In the context of natural language processing and information retrieval, ontologies can improve the results of the word sense disambiguation (WSD) techniques. By making explicit the semantics of the term, ontology-based semantic measures play a crucial role in determining how different ontology classes have a similar or related meaning. In this context, it is common to use semantic similarity as a basis for WSD. However, the measures generally consider only taxonomic relationships, which negatively affect the discrimination of two ontology classes that are related by the other relationship types. On the other hand, semantic relatedness measures consider diverse types of relationships to determine how much two classes on the ontology are related. However, these measures, especially the path-based approaches, have as the main drawback a high computational complexity to calculate the relatedness value. Also, for both types of semantic measures, it is unpractical to store all similarity or relatedness values between all ontology classes in memory, especially for ontologies with a large number of classes. In this work, we propose a novel approach based on semantic neighbors that aim to improve the performance of the knowledge-based measures in relatedness analysis. We also explain how to use this proposal into the path and feature-based measures. We evaluate our proposal on WSD using an existent domain ontology for a well-core description. This ontology contains 929 classes related to rock facies. Also, we use a set of sentences from four different corpora on the Oil&Gas domain. In the experiments, we compare our proposal with state-of-the-art semantic relatedness measures, such as path-based, feature-based, information content, and hybrid methods regarding the F-score, evaluation time, and memory consumption. The experimental results show that the proposed method obtains F-score gains in WSD, as well as a low evaluation time and memory consumption concerning the traditional knowledge-based measures.No contexto do processamento de linguagem natural e recuperação de informações, as ontologias podem melhorar os resultados das técnicas de desambiguação. Ao tornar explícita a semântica do termo, as medidas semânticas baseadas em ontologia desempenham um papel crucial para determinar como diferentes classes de ontologia têm um significado semelhante ou relacionado. Nesse contexto, é comum usar similaridade semântica como base para a desembiguação. No entanto, as medidas geralmente consideram apenas relações taxonômicas, o que afeta negativamente a discriminação de duas classes de ontologia relacionadas por outros tipos de relações. Por outro lado, as medidas de relacionamento semântico consideram diversos tipos de relacionamentos ontológicos para determinar o quanto duas classes estão relacionadas. No entanto, essas medidas, especialmente as abordagens baseadas em caminhos, têm como principal desvantagem uma alta complexidade computacional para sua execução. Além disso, tende a ser impraticável armazenar na memória todos os valores de similaridade ou relacionamento entre todas as classes de uma ontologia, especialmente para ontologias com um grande número de classes. Neste trabalho, propomos uma nova abordagem baseada em vizinhos semânticos que visa melhorar o desempenho das medidas baseadas em conhecimento na análise de relacionamento. Também explicamos como usar esta proposta em medidas baseadas em caminhos e características. Avaliamos nossa proposta na desambiguação utilizando uma ontologia de domínio preexistente para descrição de testemunhos. Esta ontologia contém 929 classes relacionadas a fácies de rocha. Além disso, usamos um conjunto de sentenças de quatro corpora diferentes no domínio Petróleo e Gás. Em nossos experimentos, comparamos nossa proposta com medidas de relacionamento semântico do estado-daarte, como métodos baseados em caminhos, características, conteúdo de informação, e métodos híbridos em relação ao F-score, tempo de avaliação e consumo de memória. Os resultados experimentais mostram que o método proposto obtém ganhos de F-score na desambiguação, além de um baixo tempo de avaliação e consumo de memória em relação às medidas tradicionais baseadas em conhecimento

Proceedings of the Workshop Semantic Content Acquisition and Representation (SCAR) 2007

This is the proceedings of the Workshop on Semantic Content Acquisition and Representation, held in conjunction with NODALIDA 2007, on May 24 2007 in Tartu, Estonia.</p

Data-driven Synset Induction and Disambiguation for Wordnet Development

International audienceAutomatic methods for wordnet development in languages other than English generally exploit information found in Princeton WordNet (PWN) and translations extracted from parallel corpora. A common approach consists in preserving the structure of PWN and transferring its content in new languages using alignments, possibly combined with information extracted from multilingual semantic resources. Even if the role of PWN remains central in this process, these automatic methods offer an alternative to the manual elaboration of new wordnets. However, their limited coverage has a strong impact on that of the resulting resources. Following this line of research, we apply a cross-lingual word sense disambiguation method to wordnet development. Our approach exploits the output of a data-driven sense induction method that generates sense clusters in new languages, similar to wordnet synsets, by identifying word senses and relations in parallel corpora. We apply our cross-lingual word sense disambiguation method to the task of enriching a French wordnet resource, the WOLF, and show how it can be efficiently used for increasing its coverage. Although our experiments involve the English-French language pair, the proposed methodology is general enough to be applied to the development of wordnet resources in other languages for which parallel corpora are available. Finally, we show how the disambiguation output can serve to reduce the granularity of new wordnets and the degree of polysemy present in PWN

Wikification of learning objects using metadata as an alternative context for disambiguation

We present a methodology to wikify learning objects. Our proposal is focused on two processes: word sense disambiguation and relevant phrase selection. The disambiguation process involves the use of the learning objects metadata as either additional or alternative context. This increases the probability of success when a learning object has a low quality context. The selection of relevant phrases is perf ormed by identifying the highest values of semantic relat edness between the main subject of a learning object and t he phrases. This criterion is useful for achieving the didactic objectives of the learning object

Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

This paper demonstrates that word sense disambiguation (WSD) can improve neural machine translation (NMT) by widening the source context considered when modeling the senses of potentially ambiguous words. We first introduce three adaptive clustering algorithms for WSD, based on k-means, Chinese restaurant processes, and random walks, which are then applied to large word contexts represented in a low-rank space and evaluated on SemEval shared-task data. We then learn word vectors jointly with sense vectors defined by our best WSD method, within a state-of-the-art NMT system. We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones. This is demonstrated by translation on five language pairs. The improvements are above one BLEU point over strong NMT baselines, +4% accuracy over all ambiguous nouns and verbs, or +20% when scored manually over several challenging words.Comment: To appear in TAC

From Word to Sense Embeddings: A Survey on Vector Representations of Meaning

Over the past years, distributed semantic representations have proved to be effective and flexible keepers of prior knowledge to be integrated into downstream applications. This survey focuses on the representation of meaning. We start from the theoretical background behind word vector space models and highlight one of their major limitations: the meaning conflation deficiency, which arises from representing a word with all its possible meanings as a single vector. Then, we explain how this deficiency can be addressed through a transition from the word level to the more fine-grained level of word senses (in its broader acceptation) as a method for modelling unambiguous lexical meaning. We present a comprehensive overview of the wide range of techniques in the two main branches of sense representation, i.e., unsupervised and knowledge-based. Finally, this survey covers the main evaluation procedures and applications for this type of representation, and provides an analysis of four of its important aspects: interpretability, sense granularity, adaptability to different domains and compositionality.Comment: 46 pages, 8 figures. Published in Journal of Artificial Intelligence Researc