428 research outputs found
Supervised Dictionary Learning and Sparse Representation-A Review
Dictionary learning and sparse representation (DLSR) is a recent and
successful mathematical model for data representation that achieves
state-of-the-art performance in various fields such as pattern recognition,
machine learning, computer vision, and medical imaging. The original
formulation for DLSR is based on the minimization of the reconstruction error
between the original signal and its sparse representation in the space of the
learned dictionary. Although this formulation is optimal for solving problems
such as denoising, inpainting, and coding, it may not lead to optimal solution
in classification tasks, where the ultimate goal is to make the learned
dictionary and corresponding sparse representation as discriminative as
possible. This motivated the emergence of a new category of techniques, which
is appropriately called supervised dictionary learning and sparse
representation (S-DLSR), leading to more optimal dictionary and sparse
representation in classification tasks. Despite many research efforts for
S-DLSR, the literature lacks a comprehensive view of these techniques, their
connections, advantages and shortcomings. In this paper, we address this gap
and provide a review of the recently proposed algorithms for S-DLSR. We first
present a taxonomy of these algorithms into six categories based on the
approach taken to include label information into the learning of the dictionary
and/or sparse representation. For each category, we draw connections between
the algorithms in this category and present a unified framework for them. We
then provide guidelines for applied researchers on how to represent and learn
the building blocks of an S-DLSR solution based on the problem at hand. This
review provides a broad, yet deep, view of the state-of-the-art methods for
S-DLSR and allows for the advancement of research and development in this
emerging area of research
Learning Discriminative Multilevel Structured Dictionaries for Supervised Image Classification
Sparse representations using overcomplete dictionaries have proved to be a
powerful tool in many signal processing applications such as denoising,
super-resolution, inpainting, compression or classification. The sparsity of
the representation very much depends on how well the dictionary is adapted to
the data at hand. In this paper, we propose a method for learning structured
multilevel dictionaries with discriminative constraints to make them well
suited for the supervised pixelwise classification of images. A multilevel
tree-structured discriminative dictionary is learnt for each class, with a
learning objective concerning the reconstruction errors of the image patches
around the pixels over each class-representative dictionary. After the initial
assignment of the class labels to image pixels based on their sparse
representations over the learnt dictionaries, the final classification is
achieved by smoothing the label image with a graph cut method and an erosion
method. Applied to a common set of texture images, our supervised
classification method shows competitive results with the state of the art
Greedy Deep Dictionary Learning
In this work we propose a new deep learning tool called deep dictionary
learning. Multi-level dictionaries are learnt in a greedy fashion, one layer at
a time. This requires solving a simple (shallow) dictionary learning problem,
the solution to this is well known. We apply the proposed technique on some
benchmark deep learning datasets. We compare our results with other deep
learning tools like stacked autoencoder and deep belief network; and state of
the art supervised dictionary learning tools like discriminative KSVD and label
consistent KSVD. Our method yields better results than all
Discriminative Nonlinear Analysis Operator Learning: When Cosparse Model Meets Image Classification
Linear synthesis model based dictionary learning framework has achieved
remarkable performances in image classification in the last decade. Behaved as
a generative feature model, it however suffers from some intrinsic
deficiencies. In this paper, we propose a novel parametric nonlinear analysis
cosparse model (NACM) with which a unique feature vector will be much more
efficiently extracted. Additionally, we derive a deep insight to demonstrate
that NACM is capable of simultaneously learning the task adapted feature
transformation and regularization to encode our preferences, domain prior
knowledge and task oriented supervised information into the features. The
proposed NACM is devoted to the classification task as a discriminative feature
model and yield a novel discriminative nonlinear analysis operator learning
framework (DNAOL). The theoretical analysis and experimental performances
clearly demonstrate that DNAOL will not only achieve the better or at least
competitive classification accuracies than the state-of-the-art algorithms but
it can also dramatically reduce the time complexities in both training and
testing phases.Comment: IEEE TIP Accepte
Structured Dictionary Learning for Classification
Sparsity driven signal processing has gained tremendous popularity in the
last decade. At its core, the assumption is that the signal of interest is
sparse with respect to either a fixed transformation or a signal dependent
dictionary. To better capture the data characteristics, various dictionary
learning methods have been proposed for both reconstruction and classification
tasks. For classification particularly, most approaches proposed so far have
focused on designing explicit constraints on the sparse code to improve
classification accuracy while simply adopting -norm or -norm for
sparsity regularization. Motivated by the success of structured sparsity in the
area of Compressed Sensing, we propose a structured dictionary learning
framework (StructDL) that incorporates the structure information on both group
and task levels in the learning process. Its benefits are two-fold: (i) the
label consistency between dictionary atoms and training data are implicitly
enforced; and (ii) the classification performance is more robust in the cases
of a small dictionary size or limited training data than other techniques.
Using the subspace model, we derive the conditions for StructDL to guarantee
the performance and show theoretically that StructDL is superior to -norm
or -norm regularized dictionary learning for classification. Extensive
experiments have been performed on both synthetic simulations and real world
applications, such as face recognition and object classification, to
demonstrate the validity of the proposed DL framework
Jointly Learning Non-negative Projection and Dictionary with Discriminative Graph Constraints for Classification
Sparse coding with dictionary learning (DL) has shown excellent
classification performance. Despite the considerable number of existing works,
how to obtain features on top of which dictionaries can be better learned
remains an open and interesting question. Many current prevailing DL methods
directly adopt well-performing crafted features. While such strategy may
empirically work well, it ignores certain intrinsic relationship between
dictionaries and features. We propose a framework where features and
dictionaries are jointly learned and optimized. The framework, named joint
non-negative projection and dictionary learning (JNPDL), enables interaction
between the input features and the dictionaries. The non-negative projection
leads to discriminative parts-based object features while DL seeks a more
suitable representation. Discriminative graph constraints are further imposed
to simultaneously maximize intra-class compactness and inter-class
separability. Experiments on both image and image set classification show the
excellent performance of JNPDL by outperforming several state-of-the-art
approaches.Comment: To appear in BMVC 201
Joint Projection and Dictionary Learning using Low-rank Regularization and Graph Constraints
In this paper, we aim at learning simultaneously a discriminative dictionary
and a robust projection matrix from noisy data. The joint learning, makes the
learned projection and dictionary a better fit for each other, so a more
accurate classification can be obtained. However, current prevailing joint
dimensionality reduction and dictionary learning methods, would fail when the
training samples are noisy or heavily corrupted. To address this issue, we
propose a joint projection and dictionary learning using low-rank
regularization and graph constraints (JPDL-LR). Specifically, the
discrimination of the dictionary is achieved by imposing Fisher criterion on
the coding coefficients. In addition, our method explicitly encodes the local
structure of data by incorporating a graph regularization term, that further
improves the discriminative ability of the projection matrix. Inspired by
recent advances of low-rank representation for removing outliers and noise, we
enforce a low-rank constraint on sub-dictionaries of all classes to make them
more compact and robust to noise. Experimental results on several benchmark
datasets verify the effectiveness and robustness of our method for both
dimensionality reduction and image classification, especially when the data
contains considerable noise or variations
Discriminative Bayesian Dictionary Learning for Classification
We propose a Bayesian approach to learn discriminative dictionaries for
sparse representation of data. The proposed approach infers probability
distributions over the atoms of a discriminative dictionary using a Beta
Process. It also computes sets of Bernoulli distributions that associate class
labels to the learned dictionary atoms. This association signifies the
selection probabilities of the dictionary atoms in the expansion of
class-specific data. Furthermore, the non-parametric character of the proposed
approach allows it to infer the correct size of the dictionary. We exploit the
aforementioned Bernoulli distributions in separately learning a linear
classifier. The classifier uses the same hierarchical Bayesian model as the
dictionary, which we present along the analytical inference solution for Gibbs
sampling. For classification, a test instance is first sparsely encoded over
the learned dictionary and the codes are fed to the classifier. We performed
experiments for face and action recognition; and object and scene-category
classification using five public datasets and compared the results with
state-of-the-art discriminative sparse representation approaches. Experiments
show that the proposed Bayesian approach consistently outperforms the existing
approaches.Comment: 15 page
A Study on Unsupervised Dictionary Learning and Feature Encoding for Action Classification
Many efforts have been devoted to develop alternative methods to traditional
vector quantization in image domain such as sparse coding and soft-assignment.
These approaches can be split into a dictionary learning phase and a feature
encoding phase which are often closely connected. In this paper, we investigate
the effects of these phases by separating them for video-based action
classification. We compare several dictionary learning methods and feature
encoding schemes through extensive experiments on KTH and HMDB51 datasets.
Experimental results indicate that sparse coding performs consistently better
than the other encoding methods in large complex dataset (i.e., HMDB51), and it
is robust to different dictionaries. For small simple dataset (i.e., KTH) with
less variation, however, all the encoding strategies perform competitively. In
addition, we note that the strength of sophisticated encoding approaches comes
not from their corresponding dictionaries but the encoding mechanisms, and we
can just use randomly selected exemplars as dictionaries for video-based action
classification
Scalable Block-Diagonal Locality-Constrained Projective Dictionary Learning
We propose a novel structured discriminative block-diagonal dictionary
learning method, referred to as scalable Locality-Constrained Projective
Dictionary Learning (LC-PDL), for efficient representation and classification.
To improve the scalability by saving both training and testing time, our LC-PDL
aims at learning a structured discriminative dictionary and a block-diagonal
representation without using costly l0/l1-norm. Besides, it avoids extra
time-consuming sparse reconstruction process with the well-trained dictionary
for new sample as many existing models. More importantly, LC-PDL avoids using
the complementary data matrix to learn the sub-dictionary over each class. To
enhance the performance, we incorporate a locality constraint of atoms into the
DL procedures to keep local information and obtain the codes of samples over
each class separately. A block-diagonal discriminative approximation term is
also derived to learn a discriminative projection to bridge data with their
codes by extracting the special block-diagonal features from data, which can
ensure the approximate coefficients to associate with its label information
clearly. Then, a robust multiclass classifier is trained over extracted
block-diagonal codes for accurate label predictions. Experimental results
verify the effectiveness of our algorithm.Comment: Accepted at the 28th International Joint Conference on Artificial
Intelligence(IJCAI 2019
- …