467 research outputs found
Scalable Block-Diagonal Locality-Constrained Projective Dictionary Learning
We propose a novel structured discriminative block-diagonal dictionary
learning method, referred to as scalable Locality-Constrained Projective
Dictionary Learning (LC-PDL), for efficient representation and classification.
To improve the scalability by saving both training and testing time, our LC-PDL
aims at learning a structured discriminative dictionary and a block-diagonal
representation without using costly l0/l1-norm. Besides, it avoids extra
time-consuming sparse reconstruction process with the well-trained dictionary
for new sample as many existing models. More importantly, LC-PDL avoids using
the complementary data matrix to learn the sub-dictionary over each class. To
enhance the performance, we incorporate a locality constraint of atoms into the
DL procedures to keep local information and obtain the codes of samples over
each class separately. A block-diagonal discriminative approximation term is
also derived to learn a discriminative projection to bridge data with their
codes by extracting the special block-diagonal features from data, which can
ensure the approximate coefficients to associate with its label information
clearly. Then, a robust multiclass classifier is trained over extracted
block-diagonal codes for accurate label predictions. Experimental results
verify the effectiveness of our algorithm.Comment: Accepted at the 28th International Joint Conference on Artificial
Intelligence(IJCAI 2019
Locality Constraint Dictionary Learning with Support Vector for Pattern Classification
Discriminative dictionary learning (DDL) has recently gained significant
attention due to its impressive performance in various pattern classification
tasks. However, the locality of atoms is not fully explored in conventional DDL
approaches which hampers their classification performance. In this paper, we
propose a locality constraint dictionary learning with support vector
discriminative term (LCDL-SV), in which the locality information is preserved
by employing the graph Laplacian matrix of the learned dictionary. To jointly
learn a classifier during the training phase, a support vector discriminative
term is incorporated into the proposed objective function. Moreover, in the
classification stage, the identity of test data is jointly determined by the
regularized residual and the learned multi-class support vector machine.
Finally, the resulting optimization problem is solved by utilizing the
alternative strategy. Experimental results on benchmark databases demonstrate
the superiority of our proposed method over previous dictionary learning
approaches on both hand-crafted and deep features. The source code of our
proposed LCDL-SV is accessible at https://github.com/yinhefeng/LCDL-SVComment: submitted to IEEE Acces
When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data
We present a new Deep Dictionary Learning and Coding Network (DDLCN) for
image recognition tasks with limited data. The proposed DDLCN has most of the
standard deep learning layers (e.g., input/output, pooling, fully connected,
etc.), but the fundamental convolutional layers are replaced by our proposed
compound dictionary learning and coding layers. The dictionary learning learns
an over-complete dictionary for input training data. At the deep coding layer,
a locality constraint is added to guarantee that the activated dictionary bases
are close to each other. Then the activated dictionary atoms are assembled and
passed to the compound dictionary learning and coding layers. In this way, the
activated atoms in the first layer can be represented by the deeper atoms in
the second dictionary. Intuitively, the second dictionary is designed to learn
the fine-grained components shared among the input dictionary atoms, thus a
more informative and discriminative low-level representation of the dictionary
atoms can be obtained. We empirically compare DDLCN with several leading
dictionary learning methods and deep learning models. Experimental results on
five popular datasets show that DDLCN achieves competitive results compared
with state-of-the-art methods when the training data is limited. Code is
available at https://github.com/Ha0Tang/DDLCN.Comment: Accepted to TNNLS, an extended version of a paper published in
WACV2019. arXiv admin note: substantial text overlap with arXiv:1809.0418
A survey of sparse representation: algorithms and applications
Sparse representation has attracted much attention from researchers in fields
of signal processing, image processing, computer vision and pattern
recognition. Sparse representation also has a good reputation in both
theoretical research and practical applications. Many different algorithms have
been proposed for sparse representation. The main purpose of this article is to
provide a comprehensive study and an updated review on sparse representation
and to supply a guidance for researchers. The taxonomy of sparse representation
methods can be studied from various viewpoints. For example, in terms of
different norm minimizations used in sparsity constraints, the methods can be
roughly categorized into five groups: sparse representation with -norm
minimization, sparse representation with -norm (0p1) minimization,
sparse representation with -norm minimization and sparse representation
with -norm minimization. In this paper, a comprehensive overview of
sparse representation is provided. The available sparse representation
algorithms can also be empirically categorized into four groups: greedy
strategy approximation, constrained optimization, proximity algorithm-based
optimization, and homotopy algorithm-based sparse representation. The
rationales of different algorithms in each category are analyzed and a wide
range of sparse representation applications are summarized, which could
sufficiently reveal the potential nature of the sparse representation theory.
Specifically, an experimentally comparative study of these sparse
representation algorithms was presented. The Matlab code used in this paper can
be available at: http://www.yongxu.org/lunwen.html.Comment: Published on IEEE Access, Vol. 3, pp. 490-530, 201
Joint Label Prediction based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation
Constrained Concept Factorization (CCF) yields the enhanced representation
ability over CF by incorporating label information as additional constraints,
but it cannot classify and group unlabeled data appropriately. Minimizing the
difference between the original data and its reconstruction directly can enable
CCF to model a small noisy perturbation, but is not robust to gross sparse
errors. Besides, CCF cannot preserve the manifold structures in new
representation space explicitly, especially in an adaptive manner. In this
paper, we propose a joint label prediction based Robust Semi-Supervised
Adaptive Concept Factorization (RS2ACF) framework. To obtain robust
representation, RS2ACF relaxes the factorization to make it simultaneously
stable to small entrywise noise and robust to sparse errors. To enrich prior
knowledge to enhance the discrimination, RS2ACF clearly uses class information
of labeled data and more importantly propagates it to unlabeled data by jointly
learning an explicit label indicator for unlabeled data. By the label
indicator, RS2ACF can ensure the unlabeled data of the same predicted label to
be mapped into the same class in feature space. Besides, RS2ACF incorporates
the joint neighborhood reconstruction error over the new representations and
predicted labels of both labeled and unlabeled data, so the manifold structures
can be preserved explicitly and adaptively in the representation space and
label space at the same time. Owing to the adaptive manner, the tricky process
of determining the neighborhood size or kernel width can be avoided. Extensive
results on public databases verify that our RS2ACF can deliver state-of-the-art
data representation, compared with other related methods.Comment: Accepted at IEEE TKD
Learning Structured Twin-Incoherent Twin-Projective Latent Dictionary Pairs for Classification
In this paper, we extend the popular dictionary pair learning (DPL) into the
scenario of twin-projective latent flexible DPL under a structured
twin-incoherence. Technically, a novel framework called Twin-Projective Latent
Flexible DPL (TP-DPL) is proposed, which minimizes the twin-incoherence
constrained flexibly-relaxed reconstruction error to avoid the possible
over-fitting issue and produce accurate reconstruction. In this setting, our
TP-DPL integrates the twin-incoherence based latent flexible DPL and the joint
embedding of codes as well as salient features by twin-projection into a
unified model in an adaptive neighborhood-preserving manner. As a result,
TP-DPL unifies the salient feature extraction, representation and
classification. The twin-incoherence constraint on codes and features can
explicitly ensure high intra-class compactness and inter-class separation over
them. TP-DPL also integrates the adaptive weighting to preserve the local
neighborhood of the coefficients and salient features within each class
explicitly. For efficiency, TP-DPL uses Frobenius-norm and abandons the costly
l0/l1-norm for group sparse representation. Another byproduct is that TP-DPL
can directly apply the class-specific twin-projective reconstruction residual
to compute the label of data. Extensive results on public databases show that
TP-DPL can deliver the state-of-the-art performance.Comment: Accepted by ICDM 2019 as a regular pape
A survey of dimensionality reduction techniques
Experimental life sciences like biology or chemistry have seen in the recent
decades an explosion of the data available from experiments. Laboratory
instruments become more and more complex and report hundreds or thousands
measurements for a single experiment and therefore the statistical methods face
challenging tasks when dealing with such high dimensional data. However, much
of the data is highly redundant and can be efficiently brought down to a much
smaller number of variables without a significant loss of information. The
mathematical procedures making possible this reduction are called
dimensionality reduction techniques; they have widely been developed by fields
like Statistics or Machine Learning, and are currently a hot research topic. In
this review we categorize the plethora of dimension reduction techniques
available and give the mathematical insight behind them
Robust Unsupervised Flexible Auto-weighted Local-Coordinate Concept Factorization for Image Clustering
We investigate the high-dimensional data clustering problem by proposing a
novel and unsupervised representation learning model called Robust Flexible
Auto-weighted Local-coordinate Concept Factorization (RFA-LCF). RFA-LCF
integrates the robust flexible CF, robust sparse local-coordinate coding and
the adaptive reconstruction weighting learning into a unified model. The
adaptive weighting is driven by including the joint manifold preserving
constraints on the recovered clean data, basis concepts and new representation.
Specifically, our RFA-LCF uses a L2,1-norm based flexible residue to encode the
mismatch between clean data and its reconstruction, and also applies the robust
adaptive sparse local-coordinate coding to represent the data using a few
nearby basis concepts, which can make the factorization more accurate and
robust to noise. The robust flexible factorization is also performed in the
recovered clean data space for enhancing representations. RFA-LCF also
considers preserving the local manifold structures of clean data space, basis
concept space and the new coordinate space jointly in an adaptive manner way.
Extensive comparisons show that RFA-LCF can deliver enhanced clustering
results.Comment: Accepted at the 44th IEEE International Conference on Acoustics,
Speech, and Signal Processing(ICASSP 2019
Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier
In this paper, we propose an analysis mechanism based structured Analysis
Discriminative Dictionary Learning (ADDL) framework. ADDL seamlessly integrates
the analysis discriminative dictionary learning, analysis representation and
analysis classifier training into a unified model. The applied analysis
mechanism can make sure that the learnt dictionaries, representations and
linear classifiers over different classes are independent and discriminating as
much as possible. The dictionary is obtained by minimizing a reconstruction
error and an analytical incoherence promoting term that encourages the
sub-dictionaries associated with different classes to be independent. To obtain
the representation coefficients, ADDL imposes a sparse l2,1-norm constraint on
the coding coefficients instead of using l0 or l1-norm, since the l0 or l1-norm
constraint applied in most existing DL criteria makes the training phase time
consuming. The codes-extraction projection that bridges data with the sparse
codes by extracting special features from the given samples is calculated via
minimizing a sparse codes approximation term. Then we compute a linear
classifier based on the approximated sparse codes by an analysis mechanism to
simultaneously consider the classification and representation powers. Thus, the
classification approach of our model is very efficient, because it can avoid
the extra time-consuming sparse reconstruction process with trained dictionary
for each new test data as most existing DL algorithms. Simulations on real
image databases demonstrate that our ADDL model can obtain superior performance
over other state-of-the-arts.Comment: Accepted by IEEE TNNL
Discriminative Local Sparse Representation by Robust Adaptive Dictionary Pair Learning
In this paper, we propose a structured Robust Adaptive Dic-tionary Pair
Learning (RA-DPL) framework for the discrim-inative sparse representation
learning. To achieve powerful representation ability of the available samples,
the setting of RA-DPL seamlessly integrates the robust projective dictionary
pair learning, locality-adaptive sparse representations and discriminative
coding coefficients learning into a unified learning framework. Specifically,
RA-DPL improves existing projective dictionary pair learning in four
perspectives. First, it applies a sparse l2,1-norm based metric to encode the
recon-struction error to deliver the robust projective dictionary pairs, and
the l2,1-norm has the potential to minimize the error. Sec-ond, it imposes the
robust l2,1-norm clearly on the analysis dictionary to ensure the sparse
property of the coding coeffi-cients rather than using the costly l0/l1-norm.
As such, the robustness of the data representation and the efficiency of the
learning process are jointly considered to guarantee the effi-cacy of our
RA-DPL. Third, RA-DPL conceives a structured reconstruction weight learning
paradigm to preserve the local structures of the coding coefficients within
each class clearly in an adaptive manner, which encourages to produce the
locality preserving representations. Fourth, it also considers improving the
discriminating ability of coding coefficients and dictionary by incorporating a
discriminating function, which can ensure high intra-class compactness and
inter-class separation in the code space. Extensive experiments show that our
RA-DPL can obtain superior performance over other state-of-the-arts.Comment: Accepted by IEEE TNNL
- …