Hybrid Wavelet-Support Vector Classifiers

The Support Vector Machine (SVM) represents a new and very promising technique for machine learning tasks involving classification, regression or novelty detection. Improvements of its generalization ability can be achieved by incorporating prior knowledge of the task at hand. We propose a new hybrid algorithm consisting of signal-adapted wavelet decompositions and SVMs for waveform classification. The adaptation of the wavelet decompositions is tailormade for SVMs with radial basis functions as kernels. It allows the optimization Of the representation of the data before training the SVM and does not suffer from computationally expensive validation techniques. We assess the performance of our algorithm against the background of current concerns in medical diagnostics, namely the classification of endocardial electrograms and the detection of otoacoustic emissions. Here the performance of SVMs can significantly be improved by our adapted preprocessing step

Classification via local multi-resolution projections

We focus on the supervised binary classification problem, which consists in guessing the label $Y$ associated to a co-variate $X \in \R^d$, given a set of $n$ independent and identically distributed co-variates and associated labels $(X_i,Y_i)$. We assume that the law of the random vector $(X,Y)$ is unknown and the marginal law of $X$ admits a density supported on a set \A. In the particular case of plug-in classifiers, solving the classification problem boils down to the estimation of the regression function \eta(X) = \Exp[Y|X]. Assuming first \A to be known, we show how it is possible to construct an estimator of $\eta$ by localized projections onto a multi-resolution analysis (MRA). In a second step, we show how this estimation procedure generalizes to the case where \A is unknown. Interestingly, this novel estimation procedure presents similar theoretical performances as the celebrated local-polynomial estimator (LPE). In addition, it benefits from the lattice structure of the underlying MRA and thus outperforms the LPE from a computational standpoint, which turns out to be a crucial feature in many practical applications. Finally, we prove that the associated plug-in classifier can reach super-fast rates under a margin assumption.Comment: 38 pages, 6 figure

Effectively Finding the Optimal Wavelet for Hybrid Wavelet - Large Margin Signal Classification

For hybrid wavelet - large margin classifiers, adapting the wavelet may significantly improve the classification performance. We propose to select the wavelet with respect to a large margin classifier and data to improve class separability and minimise the generalisation error. In this paper, we show that this wavelet adaptation problem can be formulated as an optimisation problem with polynomial objective function and investigate some techniques to solve it. In particular, we propose an adaptive grid search algorithm that efficiently solves the problem compared with standard optimisation techniques

Adaptive Conjoint Wavelet-Support Vector Classifiers

Combined wavelet - large margin classifiers succeed in solving difficult signal classification problems in cases where solely using a large margin classifier like, e.g., the Support Vector Machine may fail. This thesis investigates the problem of conjointly designing both classifier stages to achieve a most effective classifier architecture. Particularly, the wavelet features should be adapted to the Support Vector classifier and the specific classification problem. Three different approaches to achieve this goal are considered: The classifier performance is seriously affected by the wavelet or filter used for feature extraction. To optimally choose this wavelet with respect to the subsequent Support Vector classification, appropriate criteria may be used. The radius - margin Support Vector Machine error bound is proven to be computable by two standard Support Vector problems. Criteria which are computationally still more efficient may be sufficient for filter adaptation. For the classification by a Support Vector Machine, several criteria are examined rating feature sets obtained from various orthogonal filter banks. An adaptive search algorithm is devised that, once the criterion is fixed, efficiently finds the optimal wavelet filter. To extract shift invariant wavelet features, Kingsbury's dual-tree complex wavelet transform is examined. The dual-tree filter bank construction leads to wavelets with vanishing negative frequency parts. An enhanced transform is established in the frequency domain for standard wavelet filters without special filter design. The translation and rotational invariance is improved compared with the common wavelet transform as shown for various standard wavelet filters. So the framework well applies to adapted signal classification. Wavelet adaptation for signal classification is a special case of feature selection. Feature selection is an important combinatorial optimisation problem in the context of supervised pattern classification. Four novel continuous feature selection approaches directly minimising the classifier performance are presented. In particular, they include linear and nonlinear Support Vector classifiers. The key ideas of the approaches are additional regularisation and embedded nonlinear feature selection. To solve the optimisation problems, difference of convex functions programming which is a general framework for non-convex continuous optimisation is applied. This optimisation framework may also be interesting for other applications and succeeds in robustly solving the problems, and hence, building more powerful feature selection methods

Hybrid Machine Learning Models for Classifying Power Quality Disturbances: A Comparative Study

The economic impact associated with power quality (PQ) problems in electrical systems is increasing, so PQ improvement research becomes a key task. In this paper, a Stockwell transform (ST)-based hybrid machine learning approach was used for the recognition and classification of power quality disturbances (PQDs). The ST of the PQDs was used to extract significant waveform features which constitute the input vectors for different machine learning approaches, including the K-nearest neighbors’ algorithm (K-NN), decision tree (DT), and support vector machine (SVM) used for classifying the PQDs. The procedure was optimized by using the genetic algorithm (GA) and the competitive swarm optimization algorithm (CSO). To test the proposed methodology, synthetic PQD waveforms were generated. Typical single disturbances for the voltage signal, as well as complex disturbances resulting from possible combinations of them, were considered. Furthermore, different levels of white Gaussian noise were added to the PQD waveforms while maintaining the desired accuracy level of the proposed classification methods. Finally, all the hybrid classification proposals were evaluated and the best one was compared with some others present in the literature. The proposed ST-based CSO-SVM method provides good results in terms of classification accuracy and noise immunity

Hybrid image representation methods for automatic image annotation: a survey

In most automatic image annotation systems, images are represented with low level features using either global methods or local methods. In global methods, the entire image is used as a unit. Local methods divide images into blocks where fixed-size sub-image blocks are adopted as sub-units; or into regions by using segmented regions as sub-units in images. In contrast to typical automatic image annotation methods that use either global or local features exclusively, several recent methods have considered incorporating the two kinds of information, and believe that the combination of the two levels of features is beneficial in annotating images. In this paper, we provide a survey on automatic image annotation techniques according to one aspect: feature extraction, and, in order to complement existing surveys in literature, we focus on the emerging image annotation methods: hybrid methods that combine both global and local features for image representation

Classification of EMG signals to control a prosthetic hand using time-frequesncy representations and Support Vector Machines

Myoelectric signals (MES) are viable control signals for externally-powered prosthetic devices. They may improve both the functionality and the cosmetic appearance of these devices. Conventional controllers, based on the signal\u27s amplitude features in the control strategy, lack a large number of controllable states because signals from independent muscles are required for each degree of freedom (DoF) of the device. Myoelectric pattern recognition systems can overcome this problem by discriminating different residual muscle movements instead of contraction levels of individual muscles. However, the lack of long-term robustness in these systems and the design of counter-intuitive control/command interfaces have resulted in low clinical acceptance levels. As a result, the development of robust, easy to use myoelectric pattern recognition-based control systems is the main challenge in the field of prosthetic control. This dissertation addresses the need to improve the controller\u27s robustness by designing a pattern recognition-based control system that classifies the user\u27s intention to actuate the prosthesis. This system is part of a cost-effective prosthetic hand prototype developed to achieve an acceptable level of functional dexterity using a simple to use interface. A Support Vector Machine (SVM) classifier implemented as a directed acyclic graph (DAG) was created. It used wavelet features from multiple surface EMG channels strategically placed over five forearm muscles. The classifiers were evaluated across seven subjects. They were able to discriminate five wrist motions with an accuracy of 91.5%. Variations of electrode locations were artificially introduced at each recording session as part of the procedure, to obtain data that accounted for the changes in the user\u27s muscle patterns over time. The generalization ability of the SVM was able to capture most of the variability in the data and to maintain an average classification accuracy of 90%. Two principal component analysis (PCA) frameworks were also evaluated to study the relationship between EMG recording sites and the need for feature space reduction. The dimension of the new feature set was reduced with the goal of improving the classification accuracy and reducing the computation time. The analysis indicated that the projection of the wavelet features into a reduced feature space did not significantly improve the accuracy and the computation time. However, decreasing the number of wavelet decomposition levels did lower the computational load without compromising the average signal classification accuracy. Based on the results of this work, a myoelectric pattern recognition-based control system that uses an SVM classifier applied to time-frequency features may be used to discriminate muscle contraction patterns for prosthetic applications

Intelligent Methods for Characterization of Electrical Power Quality Signals using Higher Order Statistical Features

This paper considers a few important techniques classification for to identify several power quality disturbances. For this purpose, a process based in HOS has been realized to extract features that help in classification. In this stage the geometrical pattern established via higher-order statistical measurements is obtained, and this pattern is function of the amplitudes and frequencies of the power quality disturbances associated to the 50-Hz power-line. Once the features are managed will be segmented to form training and test sets and them will be applied in the statistical methods used to perform automatic classification of PQ disturbances. The best technique of those compared is selected according to correlation and mistake rates