Dictionary Learning for Signal Classification.

Signal classification is widely applied in science and engineering such as in audio and visual signal processing. The performance of a typical classification system depends highly on the features (used to represent a signal in a lower dimensional space) and the classification algorithms (used to determine the category of the signal based on the features). Recent developments show that dictionary learning based sparse representation techniques have the potential to offer improved performance over the conventional techniques for feature extraction, such as mel frequency cepstrum coefficient (MFCC) and classifier design, such as support vector machine (SVM). In this thesis, we focus on dictionary learning based methods for signal classification and address several challenges as explained below. First we study the potential of using dictionary learning algorithms such as K-SVD for sparse feature extraction obtained by Orthogonal Matching Pursuit (OMP). Specifically, we have proposed the use of pooling and sampling techniques in audio domain to unify the dimension of feature vectors, and to improve computational efficiency. The proposed algorithm is also shown to have advantages for noisy signal classification. Most dictionary learning algorithms have been developed for vector/matrix form of data. Our second contribution is to extend dictionary learning algorithms for high dimensional tensor data and use them to design classifiers. Different from existing tensor dictionary learning methods, we introduce various constraints on the dictionary learning process such as structured sparsity constraints on the core tensor and discriminative constraints on the dictionaries based on the data-spread information measured by Fisher criterion. Such constraints facilitate the design of discriminative classifiers based on reconstruction error and further improve the overall performance even with reduced amount of training data. Recently, structured block sparsity in vector/matrix based dictionary learning method has been shown to outperform signal classification in terms of non-block sparse reconstruction error. In our third contribution, we extend the concept of structured-block sparsity to tensors by providing underlying dictionaries with block structure. We develop an algorithm for structured block-sparse tensor representation and perform classification based upon the block sparse tensor reconstruction error. Our algorithm shows improved performance over its matrix based counter-parts and comparable performance with our previous tensor based method. Our dictionary learning based classification methods are applied on audio and image data for various application scenarios such as speech and music discrimination, speaker identification, digit and face recognition. The experimental results confirm the advantage of the proposed algorithms over several state-of-the-art baseline algorithms

Dictionary Learning for Signal Classification.

Signal classification is widely applied in science and engineering such as in audio and visual signal processing. The performance of a typical classification system depends highly on the features (used to represent a signal in a lower dimensional space) and the classification algorithms (used to determine the category of the signal based on the features). Recent developments show that dictionary learning based sparse representation techniques have the potential to offer improved performance over the conventional techniques for feature extraction, such as mel frequency cepstrum coefficient (MFCC) and classifier design, such as support vector machine (SVM). In this thesis, we focus on dictionary learning based methods for signal classification and address several challenges as explained below. First we study the potential of using dictionary learning algorithms such as K-SVD for sparse feature extraction obtained by Orthogonal Matching Pursuit (OMP). Specifically, we have proposed the use of pooling and sampling techniques in audio domain to unify the dimension of feature vectors, and to improve computational efficiency. The proposed algorithm is also shown to have advantages for noisy signal classification. Most dictionary learning algorithms have been developed for vector/matrix form of data. Our second contribution is to extend dictionary learning algorithms for high dimensional tensor data and use them to design classifiers. Different from existing tensor dictionary learning methods, we introduce various constraints on the dictionary learning process such as structured sparsity constraints on the core tensor and discriminative constraints on the dictionaries based on the data-spread information measured by Fisher criterion. Such constraints facilitate the design of discriminative classifiers based on reconstruction error and further improve the overall performance even with reduced amount of training data. Recently, structured block sparsity in vector/matrix based dictionary learning method has been shown to outperform signal classification in terms of non-block sparse reconstruction error. In our third contribution, we extend the concept of structured-block sparsity to tensors by providing underlying dictionaries with block structure. We develop an algorithm for structured block-sparse tensor representation and perform classification based upon the block sparse tensor reconstruction error. Our algorithm shows improved performance over its matrix based counter-parts and comparable performance with our previous tensor based method. Our dictionary learning based classification methods are applied on audio and image data for various application scenarios such as speech and music discrimination, speaker identification, digit and face recognition. The experimental results confirm the advantage of the proposed algorithms over several state-of-the-art baseline algorithms