31 research outputs found
Active learning in annotating micro-blogs dealing with e-reputation
Elections unleash strong political views on Twitter, but what do people
really think about politics? Opinion and trend mining on micro blogs dealing
with politics has recently attracted researchers in several fields including
Information Retrieval and Machine Learning (ML). Since the performance of ML
and Natural Language Processing (NLP) approaches are limited by the amount and
quality of data available, one promising alternative for some tasks is the
automatic propagation of expert annotations. This paper intends to develop a
so-called active learning process for automatically annotating French language
tweets that deal with the image (i.e., representation, web reputation) of
politicians. Our main focus is on the methodology followed to build an original
annotated dataset expressing opinion from two French politicians over time. We
therefore review state of the art NLP-based ML algorithms to automatically
annotate tweets using a manual initiation step as bootstrap. This paper focuses
on key issues about active learning while building a large annotated data set
from noise. This will be introduced by human annotators, abundance of data and
the label distribution across data and entities. In turn, we show that Twitter
characteristics such as the author's name or hashtags can be considered as the
bearing point to not only improve automatic systems for Opinion Mining (OM) and
Topic Classification but also to reduce noise in human annotations. However, a
later thorough analysis shows that reducing noise might induce the loss of
crucial information.Comment: Journal of Interdisciplinary Methodologies and Issues in Science -
Vol 3 - Contextualisation digitale - 201
Learning to Sample: an Active Learning Framework
Meta-learning algorithms for active learning are emerging as a promising
paradigm for learning the ``best'' active learning strategy. However, current
learning-based active learning approaches still require sufficient training
data so as to generalize meta-learning models for active learning. This is
contrary to the nature of active learning which typically starts with a small
number of labeled samples. The unavailability of large amounts of labeled
samples for training meta-learning models would inevitably lead to poor
performance (e.g., instabilities and overfitting). In our paper, we tackle
these issues by proposing a novel learning-based active learning framework,
called Learning To Sample (LTS). This framework has two key components: a
sampling model and a boosting model, which can mutually learn from each other
in iterations to improve the performance of each other. Within this framework,
the sampling model incorporates uncertainty sampling and diversity sampling
into a unified process for optimization, enabling us to actively select the
most representative and informative samples based on an optimized integration
of uncertainty and diversity. To evaluate the effectiveness of the LTS
framework, we have conducted extensive experiments on three different
classification tasks: image classification, salary level prediction, and entity
resolution. The experimental results show that our LTS framework significantly
outperforms all the baselines when the label budget is limited, especially for
datasets with highly imbalanced classes. In addition to this, our LTS framework
can effectively tackle the cold start problem occurring in many existing active
learning approaches.Comment: Accepted by ICDM'1
Generalized Query-Based Active Learning to Identify Differentially Methylated Regions in DNA
Active learning is a supervised learning technique that reduces the number of examples required for building a successful classifier, because it can choose the data it learns from. This technique holds promise for many biological domains in which classified examples are expensive and time-consuming to obtain. Most traditional active learning methods ask very specific queries to the Oracle (e.g., a human expert) to label an unlabeled example. The example may consist of numerous features, many of which are irrelevant. Removing such features will create a shorter query with only relevant features, and it will be easier for the Oracle to answer. We propose a generalized query-based active learning (GQAL) approach that constructs generalized queries based on multiple instances. By constructing appropriately generalized queries, we can achieve higher accuracy compared to traditional active learning methods. We apply our active learning method to find differentially DNA methylated regions (DMRs). DMRs are DNA locations in the genome that are known to be involved in tissue differentiation, epigenetic regulation, and disease. We also apply our method on 13 other data sets and show that our method is better than another popular active learning technique