51,627 research outputs found
Joint Active Learning with Feature Selection via CUR Matrix Decomposition
This paper presents an unsupervised learning approach for simultaneous sample
and feature selection, which is in contrast to existing works which mainly
tackle these two problems separately. In fact the two tasks are often
interleaved with each other: noisy and high-dimensional features will bring
adverse effect on sample selection, while informative or representative samples
will be beneficial to feature selection. Specifically, we propose a framework
to jointly conduct active learning and feature selection based on the CUR
matrix decomposition. From the data reconstruction perspective, both the
selected samples and features can best approximate the original dataset
respectively, such that the selected samples characterized by the features are
highly representative. In particular, our method runs in one-shot without the
procedure of iterative sample selection for progressive labeling. Thus, our
model is especially suitable when there are few labeled samples or even in the
absence of supervision, which is a particular challenge for existing methods.
As the joint learning problem is NP-hard, the proposed formulation involves a
convex but non-smooth optimization problem. We solve it efficiently by an
iterative algorithm, and prove its global convergence. Experimental results on
publicly available datasets corroborate the efficacy of our method compared
with the state-of-the-art.Comment: Accepted by T-PAM
Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision
Supervised machine learning based state-of-the-art computer vision techniques
are in general data hungry. Their data curation poses the challenges of
expensive human labeling, inadequate computing resources and larger experiment
turn around times. Training data subset selection and active learning
techniques have been proposed as possible solutions to these challenges. A
special class of subset selection functions naturally model notions of
diversity, coverage and representation and can be used to eliminate redundancy
thus lending themselves well for training data subset selection. They can also
help improve the efficiency of active learning in further reducing human
labeling efforts by selecting a subset of the examples obtained using the
conventional uncertainty sampling based techniques. In this work, we
empirically demonstrate the effectiveness of two diversity models, namely the
Facility-Location and Dispersion models for training-data subset selection and
reducing labeling effort. We demonstrate this across the board for a variety of
computer vision tasks including Gender Recognition, Face Recognition, Scene
Recognition, Object Detection and Object Recognition. Our results show that
diversity based subset selection done in the right way can increase the
accuracy by upto 5 - 10% over existing baselines, particularly in settings in
which less training data is available. This allows the training of complex
machine learning models like Convolutional Neural Networks with much less
training data and labeling costs while incurring minimal performance loss.Comment: Accepted to WACV 2019. arXiv admin note: substantial text overlap
with arXiv:1805.1119
Iterative Projection and Matching: Finding Structure-preserving Representatives and Its Application to Computer Vision
The goal of data selection is to capture the most structural information from
a set of data. This paper presents a fast and accurate data selection method,
in which the selected samples are optimized to span the subspace of all data.
We propose a new selection algorithm, referred to as iterative projection and
matching (IPM), with linear complexity w.r.t. the number of data, and without
any parameter to be tuned. In our algorithm, at each iteration, the maximum
information from the structure of the data is captured by one selected sample,
and the captured information is neglected in the next iterations by projection
on the null-space of previously selected samples. The computational efficiency
and the selection accuracy of our proposed algorithm outperform those of the
conventional methods. Furthermore, the superiority of the proposed algorithm is
shown on active learning for video action recognition dataset on UCF-101;
learning using representatives on ImageNet; training a generative adversarial
network (GAN) to generate multi-view images from a single-view input on CMU
Multi-PIE dataset; and video summarization on UTE Egocentric dataset.Comment: 11 pages, 5 figures, 5 table
A Survey on Multi-Task Learning
Multi-Task Learning (MTL) is a learning paradigm in machine learning and its
aim is to leverage useful information contained in multiple related tasks to
help improve the generalization performance of all the tasks. In this paper, we
give a survey for MTL. First, we classify different MTL algorithms into several
categories, including feature learning approach, low-rank approach, task
clustering approach, task relation learning approach, and decomposition
approach, and then discuss the characteristics of each approach. In order to
improve the performance of learning tasks further, MTL can be combined with
other learning paradigms including semi-supervised learning, active learning,
unsupervised learning, reinforcement learning, multi-view learning and
graphical models. When the number of tasks is large or the data dimensionality
is high, batch MTL models are difficult to handle this situation and online,
parallel and distributed MTL models as well as dimensionality reduction and
feature hashing are reviewed to reveal their computational and storage
advantages. Many real-world applications use MTL to boost their performance and
we review representative works. Finally, we present theoretical analyses and
discuss several future directions for MTL
Active Learning Methods based on Statistical Leverage Scores
In many real-world machine learning applications, unlabeled data are abundant
whereas class labels are expensive and scarce. An active learner aims to obtain
a model of high accuracy with as few labeled instances as possible by
effectively selecting useful examples for labeling. We propose a new selection
criterion that is based on statistical leverage scores and present two novel
active learning methods based on this criterion: ALEVS for querying single
example at each iteration and DBALEVS for querying a batch of examples. To
assess the representativeness of the examples in the pool, ALEVS and DBALEVS
use the statistical leverage scores of the kernel matrices computed on the
examples of each class. Additionally, DBALEVS selects a diverse a set of
examples that are highly representative but are dissimilar to already labeled
examples through maximizing a submodular set function defined with the
statistical leverage scores and the kernel matrix computed on the pool of the
examples. The submodularity property of the set scoring function let us
identify batches with a constant factor approximate to the optimal batch in an
efficient manner. Our experiments on diverse datasets show that querying based
on leverage scores is a powerful strategy for active learning.Comment: Submitted to Machine Learning Journal, EMLP 2019 journal trac
Dynamic Neural Network Channel Execution for Efficient Training
Existing methods for reducing the computational burden of neural networks at
run-time, such as parameter pruning or dynamic computational path selection,
focus solely on improving computational efficiency during inference. On the
other hand, in this work, we propose a novel method which reduces the memory
footprint and number of computing operations required for training and
inference. Our framework efficiently integrates pruning as part of the training
procedure by exploring and tracking the relative importance of convolutional
channels. At each training step, we select only a subset of highly salient
channels to execute according to the combinatorial upper confidence bound
algorithm, and run a forward and backward pass only on these activated
channels, hence learning their parameters. Consequently, we enable the
efficient discovery of compact models. We validate our approach empirically on
state-of-the-art CNNs - VGGNet, ResNet and DenseNet, and on several image
classification datasets. Results demonstrate our framework for dynamic channel
execution reduces computational cost up to 4x and parameter count up to 9x,
thus reducing the memory and computational demands for discovering and training
compact neural network models
Continuous Adaptation of Multi-Camera Person Identification Models through Sparse Non-redundant Representative Selection
The problem of image-base person identification/recognition is to provide an
identity to the image of an individual based on learned models that describe
his/her appearance. Most traditional person identification systems rely on
learning a static model on tediously labeled training data. Though labeling
manually is an indispensable part of a supervised framework, for a large scale
identification system labeling huge amount of data is a significant overhead.
For large multi-sensor data as typically encountered in camera networks,
labeling a lot of samples does not always mean more information, as redundant
images are labeled several times. In this work, we propose a convex
optimization based iterative framework that progressively and judiciously
chooses a sparse but informative set of samples for labeling, with minimal
overlap with previously labeled images. We also use a structure preserving
sparse reconstruction based classifier to reduce the training burden typically
seen in discriminative classifiers. The two stage approach leads to a novel
framework for online update of the classifiers involving only the incorporation
of new labeled data rather than any expensive training phase. We demonstrate
the effectiveness of our approach on multi-camera person re-identification
datasets, to demonstrate the feasibility of learning online classification
models in multi-camera big data applications. Using three benchmark datasets,
we validate our approach and demonstrate that our framework achieves superior
performance with significantly less amount of manual labeling
Nonnegative Restricted Boltzmann Machines for Parts-based Representations Discovery and Predictive Model Stabilization
The success of any machine learning system depends critically on effective
representations of data. In many cases, it is desirable that a representation
scheme uncovers the parts-based, additive nature of the data. Of current
representation learning schemes, restricted Boltzmann machines (RBMs) have
proved to be highly effective in unsupervised settings. However, when it comes
to parts-based discovery, RBMs do not usually produce satisfactory results. We
enhance such capacity of RBMs by introducing nonnegativity into the model
weights, resulting in a variant called nonnegative restricted Boltzmann machine
(NRBM). The NRBM produces not only controllable decomposition of data into
interpretable parts but also offers a way to estimate the intrinsic nonlinear
dimensionality of data, and helps to stabilize linear predictive models. We
demonstrate the capacity of our model on applications such as handwritten digit
recognition, face recognition, document classification and patient readmission
prognosis. The decomposition quality on images is comparable with or better
than what produced by the nonnegative matrix factorization (NMF), and the
thematic features uncovered from text are qualitatively interpretable in a
similar manner to that of the latent Dirichlet allocation (LDA). The stability
performance of feature selection on medical data is better than RBM and
competitive with NMF. The learned features, when used for classification, are
more discriminative than those discovered by both NMF and LDA and comparable
with those by RBM
Are all training examples equally valuable?
When learning a new concept, not all training examples may prove equally
useful for training: some may have higher or lower training value than others.
The goal of this paper is to bring to the attention of the vision community the
following considerations: (1) some examples are better than others for training
detectors or classifiers, and (2) in the presence of better examples, some
examples may negatively impact performance and removing them may be beneficial.
In this paper, we propose an approach for measuring the training value of an
example, and use it for ranking and greedily sorting examples. We test our
methods on different vision tasks, models, datasets and classifiers. Our
experiments show that the performance of current state-of-the-art detectors and
classifiers can be improved when training on a subset, rather than the whole
training set
Semidefinite Programming Based Preconditioning for More Robust Near-Separable Nonnegative Matrix Factorization
Nonnegative matrix factorization (NMF) under the separability assumption can
provably be solved efficiently, even in the presence of noise, and has been
shown to be a powerful technique in document classification and hyperspectral
unmixing. This problem is referred to as near-separable NMF and requires that
there exists a cone spanned by a small subset of the columns of the input
nonnegative matrix approximately containing all columns. In this paper, we
propose a preconditioning based on semidefinite programming making the input
matrix well-conditioned. This in turn can improve significantly the performance
of near-separable NMF algorithms which is illustrated on the popular successive
projection algorithm (SPA). The new preconditioned SPA is provably more robust
to noise, and outperforms SPA on several synthetic data sets. We also show how
an active-set method allow us to apply the preconditioning on large-scale
real-world hyperspectral images.Comment: 25 pages, 6 figures, 4 tables. New numerical experiments, additional
remarks and comment
- …