Selective Sampling for Example-based Word Sense Disambiguation

This paper proposes an efficient example sampling method for example-based word sense disambiguation systems. To construct a database of practical size, a considerable overhead for manual sense disambiguation (overhead for supervision) is required. In addition, the time complexity of searching a large-sized database poses a considerable problem (overhead for search). To counter these problems, our method selectively samples a smaller-sized effective subset from a given example set for use in word sense disambiguation. Our method is characterized by the reliance on the notion of training utility: the degree to which each example is informative for future example sampling when used for the training of the system. The system progressively collects examples by selecting those with greatest utility. The paper reports the effectiveness of our method through experiments on about one thousand sentences. Compared to experiments with other example sampling methods, our method reduced both the overhead for supervision and the overhead for search, without the degeneration of the performance of the system.Comment: 25 pages, 14 Postscript figure

Naive Bayes and Exemplar-Based approaches to Word Sense Disambiguation Revisited

This paper describes an experimental comparison between two standard supervised learning methods, namely Naive Bayes and Exemplar-based classification, on the Word Sense Disambiguation (WSD) problem. The aim of the work is twofold. Firstly, it attempts to contribute to clarify some confusing information about the comparison between both methods appearing in the related literature. In doing so, several directions have been explored, including: testing several modifications of the basic learning algorithms and varying the feature space. Secondly, an improvement of both algorithms is proposed, in order to deal with large attribute sets. This modification, which basically consists in using only the positive information appearing in the examples, allows to improve greatly the efficiency of the methods, with no loss in accuracy. The experiments have been performed on the largest sense-tagged corpus available containing the most frequent and ambiguous English words. Results show that the Exemplar-based approach to WSD is generally superior to the Bayesian approach, especially when a specific metric for dealing with symbolic attributes is used.Comment: 5 page

Boosting Applied to Word Sense Disambiguation

In this paper Schapire and Singer's AdaBoost.MH boosting algorithm is applied to the Word Sense Disambiguation (WSD) problem. Initial experiments on a set of 15 selected polysemous words show that the boosting approach surpasses Naive Bayes and Exemplar-based approaches, which represent state-of-the-art accuracy on supervised WSD. In order to make boosting practical for a real learning domain of thousands of words, several ways of accelerating the algorithm by reducing the feature space are studied. The best variant, which we call LazyBoosting, is tested on the largest sense-tagged corpus available containing 192,800 examples of the 191 most frequent and ambiguous English words. Again, boosting compares favourably to the other benchmark algorithms.Comment: 12 page

Bag-Level Aggregation for Multiple Instance Active Learning in Instance Classification Problems

A growing number of applications, e.g. video surveillance and medical image analysis, require training recognition systems from large amounts of weakly annotated data while some targeted interactions with a domain expert are allowed to improve the training process. In such cases, active learning (AL) can reduce labeling costs for training a classifier by querying the expert to provide the labels of most informative instances. This paper focuses on AL methods for instance classification problems in multiple instance learning (MIL), where data is arranged into sets, called bags, that are weakly labeled. Most AL methods focus on single instance learning problems. These methods are not suitable for MIL problems because they cannot account for the bag structure of data. In this paper, new methods for bag-level aggregation of instance informativeness are proposed for multiple instance active learning (MIAL). The \textit{aggregated informativeness} method identifies the most informative instances based on classifier uncertainty, and queries bags incorporating the most information. The other proposed method, called \textit{cluster-based aggregative sampling}, clusters data hierarchically in the instance space. The informativeness of instances is assessed by considering bag labels, inferred instance labels, and the proportion of labels that remain to be discovered in clusters. Both proposed methods significantly outperform reference methods in extensive experiments using benchmark data from several application domains. Results indicate that using an appropriate strategy to address MIAL problems yields a significant reduction in the number of queries needed to achieve the same level of performance as single instance AL methods

Viabilidade do aprendizado ativo em máquinas extremas

O aprendizado de máquina requer a indução de modelos preditivos. Frequentemente, há dois problemas relacionados com essa tarefa: o custo de rotulação e o tempo de treinamento. Isso é especialmente verdade na presença massiva de dados e em sistemas interativos ou que requeiram resposta imediata. Parte da solução é o uso de aprendizado ativo, que seleciona exemplos a rotular de acordo com um critério de relevância. O complemento da solução é a adoção de um rápido e robusto algoritmo de aprendizado, como as “máquinas extremas”. Neste artigo, várias estratégias de aprendizado ativo são comparadas experimentalmente em diferentes bases de dados com o intuito de preencher uma notável lacuna nas literaturas de aprendizado ativo e máquinas extremas. Os resultados demonstram a viabilidade da união entre as duas áreas.CAPESCNPqFAPES

Selectively inhibiting learning bias for active sampling

Efficient training of machine learning algorithms requires a reliable labeled set from the application domain. Usually, data labeling is a costly process. Therefore, a selective approach is desirable. Active learning has been successfully used to reduce the labeling effort, due to its parsimonious process of querying the labeler. Nevertheless, many active learning strategies are dependent on early predictions made by learning algorithms. This might be a major problem when the learner is still unable to provide reliable information. In this context, agnostic strategies can be convenient, since they spare internal learners - usually favoring exploratory queries. On the other hand, prospective queries could benefit from a learning bias. In this article, we highlight the advantages of the agnostic approach and propose how to explore some of them without foregoing prospection. A simple hybrid strategy and a visualization tool called ranking curves, are proposed as a proof of concept. The tool allowed to see clearly when the presence of a learner was possibly detrimental. Finally, the hybrid strategy was successfully compared to its counterpart in the literature, to pure agnostic strategies and to the usual baseline of the field.CAPESCNPqFAPES