8,817 research outputs found
Learning in Unlabeled Networks - An Active Learning and Inference Approach
The task of determining labels of all network nodes based on the knowledge
about network structure and labels of some training subset of nodes is called
the within-network classification. It may happen that none of the labels of the
nodes is known and additionally there is no information about number of classes
to which nodes can be assigned. In such a case a subset of nodes has to be
selected for initial label acquisition. The question that arises is: "labels of
which nodes should be collected and used for learning in order to provide the
best classification accuracy for the whole network?". Active learning and
inference is a practical framework to study this problem.
A set of methods for active learning and inference for within network
classification is proposed and validated. The utility score calculation for
each node based on network structure is the first step in the process. The
scores enable to rank the nodes. Based on the ranking, a set of nodes, for
which the labels are acquired, is selected (e.g. by taking top or bottom N from
the ranking). The new measure-neighbour methods proposed in the paper suggest
not obtaining labels of nodes from the ranking but rather acquiring labels of
their neighbours. The paper examines 29 distinct formulations of utility score
and selection methods reporting their impact on the results of two collective
classification algorithms: Iterative Classification Algorithm and Loopy Belief
Propagation.
We advocate that the accuracy of presented methods depends on the structural
properties of the examined network. We claim that measure-neighbour methods
will work better than the regular methods for networks with higher clustering
coefficient and worse than regular methods for networks with low clustering
coefficient. According to our hypothesis, based on clustering coefficient we
are able to recommend appropriate active learning and inference method
Bayesian Semisupervised Learning with Deep Generative Models
Neural network based generative models with discriminative components are a
powerful approach for semi-supervised learning. However, these techniques a)
cannot account for model uncertainty in the estimation of the model's
discriminative component and b) lack flexibility to capture complex stochastic
patterns in the label generation process. To avoid these problems, we first
propose to use a discriminative component with stochastic inputs for increased
noise flexibility. We show how an efficient Gibbs sampling procedure can
marginalize the stochastic inputs when inferring missing labels in this model.
Following this, we extend the discriminative component to be fully Bayesian and
produce estimates of uncertainty in its parameter values. This opens the door
for semi-supervised Bayesian active learning
Active Learning for Structured Prediction from Partially Labeled Data
We propose a general purpose active learning algorithm for structured
prediction, gathering labeled data for training a model that outputs a set of
related labels for an image or video. Active learning starts with a limited
initial training set, then iterates querying a user for labels on unlabeled
data and retraining the model. We propose a novel algorithm for selecting data
for labeling, choosing examples to maximize expected information gain based on
belief propagation inference. This is a general purpose method and can be
applied to a variety of tasks or models. As a specific example we demonstrate
this framework for learning to recognize human actions and group activities in
video sequences. Experiments show that our proposed algorithm outperforms
previous active learning methods and can achieve accuracy comparable to fully
supervised methods while utilizing significantly less labeled data
Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
Annotating the right data for training deep neural networks is an important
challenge. Active learning using uncertainty estimates from Bayesian Neural
Networks (BNNs) could provide an effective solution to this. Despite being
theoretically principled, BNNs require approximations to be applied to
large-scale problems, where both performance and uncertainty estimation are
crucial. In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a
scalable technique that uses a regularized ensemble to approximate a deep BNN.
We conduct a series of large-scale visual active learning experiments to
evaluate DPEs on classification with the CIFAR-10, CIFAR-100 and ImageNet
datasets, and semantic segmentation with the BDD100k dataset. Our models
require significantly less training data to achieve competitive performances,
and steadily improve upon strong active learning baselines as the annotation
budget is increased.Comment: arXiv admin note: text overlap with arXiv:1811.0264
Few-Shot Learning with Graph Neural Networks
We propose to study the problem of few-shot learning with the prism of
inference on a partially observed graphical model, constructed from a
collection of input images whose label can be either observed or not. By
assimilating generic message-passing inference algorithms with their
neural-network counterparts, we define a graph neural network architecture that
generalizes several of the recently proposed few-shot learning models. Besides
providing improved numerical performance, our framework is easily extended to
variants of few-shot learning, such as semi-supervised or active learning,
demonstrating the ability of graph-based models to operate well on 'relational'
tasks
Distribution Aware Active Learning
Discriminative learning machines often need a large set of labeled samples
for training. Active learning (AL) settings assume that the learner has the
freedom to ask an oracle to label its desired samples. Traditional AL
algorithms heuristically choose query samples about which the current learner
is uncertain. This strategy does not make good use of the structure of the
dataset at hand and is prone to be misguided by outliers. To alleviate this
problem, we propose to distill the structural information into a probabilistic
generative model which acts as a \emph{teacher} in our model. The active
\emph{learner} uses this information effectively at each cycle of active
learning. The proposed method is generic and does not depend on the type of
learner and teacher. We then suggest a query criterion for active learning that
is aware of distribution of data and is more robust against outliers. Our
method can be combined readily with several other query criteria for active
learning. We provide the formulation and empirically show our idea via toy and
real examples
A New Vision of Collaborative Active Learning
Active learning (AL) is a learning paradigm where an active learner has to
train a model (e.g., a classifier) which is in principal trained in a
supervised way, but in AL it has to be done by means of a data set with
initially unlabeled samples. To get labels for these samples, the active
learner has to ask an oracle (e.g., a human expert) for labels. The goal is to
maximize the performance of the model and to minimize the number of queries at
the same time. In this article, we first briefly discuss the state of the art
and own, preliminary work in the field of AL. Then, we propose the concept of
collaborative active learning (CAL). With CAL, we will overcome some of the
harsh limitations of current AL. In particular, we envision scenarios where an
expert may be wrong for various reasons, there might be several or even many
experts with different expertise, the experts may label not only samples but
also knowledge at a higher level such as rules, and we consider that the
labeling costs depend on many conditions. Moreover, in a CAL process human
experts will profit by improving their own knowledge, too.Comment: 16 pages, 6 Figure
Neural Simpletrons - Minimalistic Directed Generative Networks for Learning with Few Labels
Classifiers for the semi-supervised setting often combine strong supervised
models with additional learning objectives to make use of unlabeled data. This
results in powerful though very complex models that are hard to train and that
demand additional labels for optimal parameter tuning, which are often not
given when labeled data is very sparse. We here study a minimalistic
multi-layer generative neural network for semi-supervised learning in a form
and setting as similar to standard discriminative networks as possible. Based
on normalized Poisson mixtures, we derive compact and local learning and neural
activation rules. Learning and inference in the network can be scaled using
standard deep learning tools for parallelized GPU implementation. With the
single objective of likelihood optimization, both labeled and unlabeled data
are naturally incorporated into learning. Empirical evaluations on standard
benchmarks show, that for datasets with few labels the derived minimalistic
network improves on all classical deep learning approaches and is competitive
with their recent variants without the need of additional labels for parameter
tuning. Furthermore, we find that the studied network is the best performing
monolithic (`non-hybrid') system for few labels, and that it can be applied in
the limit of very few labels, where no other system has been reported to
operate so far
Context-Aware Query Selection for Active Learning in Event Recognition
Activity recognition is a challenging problem with many practical
applications. In addition to the visual features, recent approaches have
benefited from the use of context, e.g., inter-relationships among the
activities and objects. However, these approaches require data to be labeled,
entirely available beforehand, and not designed to be updated continuously,
which make them unsuitable for surveillance applications. In contrast, we
propose a continuous-learning framework for context-aware activity recognition
from unlabeled video, which has two distinct advantages over existing methods.
First, it employs a novel active-learning technique that not only exploits the
informativeness of the individual activities but also utilizes their contextual
information during query selection; this leads to significant reduction in
expensive manual annotation effort. Second, the learned models can be adapted
online as more data is available. We formulate a conditional random field model
that encodes the context and devise an information-theoretic approach that
utilizes entropy and mutual information of the nodes to compute the set of most
informative queries, which are labeled by a human. These labels are combined
with graphical inference techniques for incremental updates. We provide a
theoretical formulation of the active learning framework with an analytic
solution. Experiments on six challenging datasets demonstrate that our
framework achieves superior performance with significantly less manual
labeling.Comment: To appear in Transactions of Pattern Pattern Analysis and Machine
Intelligence (T-PAMI
Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable
technique that uses a regularized ensemble to approximate a deep Bayesian
Neural Network (BNN). We do so by incorporating a KL divergence penalty term
into the training objective of an ensemble, derived from the evidence lower
bound used in variational inference. We evaluate the uncertainty estimates
obtained from our models for active learning on visual classification. Our
approach steadily improves upon active learning baselines as the annotation
budget is increased.Comment: Workshop on Bayesian Deep Learning (NeurIPS 2018
- …