147,302 research outputs found
Effective Linear-Time Feature Selection
The classification learning task requires selection of a subset of features to represent patterns
to be classified. This is because the performance of the classifier and the cost of
classification are sensitive to the choice of the features used to construct the classifier.
Exhaustive search is impractical since it searches every possible combination of features.
The runtime of heuristic and random searches are better but the problem still persists
when dealing with high-dimensional datasets.
We investigate a heuristic, forward, wrapper-based approach, called Linear Sequential
Selection, which limits the search space at each iteration of the feature selection process.
We introduce randomization in the search space. The algorithm is called Randomized
Linear Sequential Selection. Our experiments demonstrate that both methods are faster,
find smaller subsets and can even increase the classification accuracy.
We also explore the idea of ensemble learning. We have proposed two ensemble creation
methods, Feature Selection Ensemble and Random Feature Ensemble. Both methods apply
a feature selection algorithm to create individual classifiers of the ensemble. Our
experiments have shown that both methods work well with high-dimensional data
Sequential approaches for learning datum-wise sparse representations
International audienceIn supervised classification, data representation is usually considered at the dataset level: one looks for the "best" representation of data assuming it to be the same for all the data in the data space. We propose a different approach where the representations used for classification are tailored to each datum in the data space. One immediate goal is to obtain sparse datum-wise representations: our approach learns to build a representation specific to each datum that contains only a small subset of the features, thus allowing classification to be fast and efficient. This representation is obtained by way of a sequential decision process that sequentially chooses which features to acquire before classifying a particular point; this process is learned through algorithms based on Reinforcement Learning. The proposed method performs well on an ensemble of medium-sized sparse classification problems. It offers an alternative to global sparsity approaches, and is a natural framework for sequential classification problems. The method extends easily to a whole family of sparsity-related problem which would otherwise require developing specific solutions. This is the case in particular for cost-sensitive and limited-budget classification, where feature acquisition is costly and is often performed sequentially. Finally, our approach can handle non-differentiable loss functions or combinatorial optimization encountered in more complex feature selection problems
Classification with Costly Features using Deep Reinforcement Learning
We study a classification problem where each feature can be acquired for a
cost and the goal is to optimize a trade-off between the expected
classification error and the feature cost. We revisit a former approach that
has framed the problem as a sequential decision-making problem and solved it by
Q-learning with a linear approximation, where individual actions are either
requests for feature values or terminate the episode by providing a
classification decision. On a set of eight problems, we demonstrate that by
replacing the linear approximation with neural networks the approach becomes
comparable to the state-of-the-art algorithms developed specifically for this
problem. The approach is flexible, as it can be improved with any new
reinforcement learning enhancement, it allows inclusion of pre-trained
high-performance classifier, and unlike prior art, its performance is robust
across all evaluated datasets.Comment: AAAI 201
Fast Video Classification via Adaptive Cascading of Deep Models
Recent advances have enabled "oracle" classifiers that can classify across
many classes and input distributions with high accuracy without retraining.
However, these classifiers are relatively heavyweight, so that applying them to
classify video is costly. We show that day-to-day video exhibits highly skewed
class distributions over the short term, and that these distributions can be
classified by much simpler models. We formulate the problem of detecting the
short-term skews online and exploiting models based on it as a new sequential
decision making problem dubbed the Online Bandit Problem, and present a new
algorithm to solve it. When applied to recognizing faces in TV shows and
movies, we realize end-to-end classification speedups of 2.4-7.8x/2.6-11.2x (on
GPU/CPU) relative to a state-of-the-art convolutional neural network, at
competitive accuracy.Comment: Accepted at IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 201
- …