Effective Features and Machine Learning Methods for Document Classification

Document classification has been involved in a variety of applications, such as phishing and fraud detection, news categorisation, and information retrieval. This thesis aims to provide novel solutions to several important problems presented by document classification. First, an improved Principal Components Analysis (PCA), based on similarity and correlation criteria instead of covariance, is proposed, which aims to capture low-dimensional feature subset that facilitates improved performance in text classification. The experimental results have demonstrated the advantages and usefulness of the proposed method for text classification in high-dimensional feature space in terms of the number of features required to achieve the best classification accuracy. Second, two hybrid feature-subset selection methods are proposed based on the combination (via either union or intersection) of the results of both supervised (in one method) and unsupervised (in the other method) filter approaches prior to the use of a wrapper, leading to low-dimensional feature subset that can achieve both high classification accuracy and good interpretability, and spend less processing time than most current methods. The experimental results have demonstrated the effectiveness of the proposed methods for feature subset selection in high-dimensional feature space in terms of the number of selected features and the processing time spent to achieve the best classification accuracy. Third, a class-specific (supervised) pre-trained approach based on a sparse autoencoder is proposed for acquiring low-dimensional interesting structure of relevant features, which can be used for high-performance document classification. The experimental results have demonstrated the merit of this proposed method for document classification in high-dimensional feature space, in terms of the limited number of features required to achieve good classification accuracy. Finally, deep classifier structures associated with a stacked autoencoder (SAE) for higher-level feature extraction are investigated, aiming to overcome the difficulties experienced in training deep neural networks with limited training data in high-dimensional feature space, such as overfitting and vanishing/exploding gradients. This investigation has resulted in a three-stage learning algorithm for training deep neural networks. In comparison with support vector machines (SVMs) combined with SAE and Deep Multilayer Perceptron (DMLP) with random weight initialisation, the experimental results have shown the advantages and effectiveness of the proposed three-stage learning algorithm

Feature extraction and selection algorithm based on self adaptive ant colony system for sky image classification

Sky image classification is crucial in meteorology to forecast weather and climatic conditions. The fine-grained cloud detection and recognition (FGCDR) algorithm is use to extract colour, inside texture and neighbour texture features from multiview of superpixels sky images. However, the FGCDR produced a substantial amount of redundant and insignificant features. The ant colony optimisation (ACO) algorithm have been used to select feature subset. However, the ACO suffers from premature convergence which leads to poor feature subset. Therefore, an improved feature extraction and selection for sky image classification (FESSIC) algorithm is proposed. This algorithm consists of (i) Gaussian smoothness standard deviation method that formulates informative features within sky images; (ii) nearest-threshold based technique that converts feature map into a weighted directed graph to represent relationship between features; and (iii) an ant colony system with self-adaptive parameter technique for local pheromone update. The performance of FESSIC was evaluated against ten benchmark image classification algorithms and six classifiers on four ground-based sky image datasets. The Friedman test result is presented for the performance rank of six benchmark feature selection algorithms and FESSIC algorithm. The Man-Whitney U test is then performed to statistically evaluate the significance difference of the second rank and FESSIC algorithms. The experimental results for the proposed algorithm are superior to the benchmark image classification algorithms in terms of similarity value on Kiel, SWIMCAT and MGCD datasets. FESSIC outperforms other algorithms for average classification accuracy for the KSVM, MLP, RF and DT classifiers. The Friedman test has shown that the FESSIC has the first rank for all classifiers. Furthermore, the result of Man-Whitney U test indicates that FESSIC is significantly better than the second rank benchmark algorithm for all classifiers. In conclusion, the FESSIC can be utilised for image classification in various applications such as disaster management, medical diagnosis, industrial inspection, sports management, and content-based image retrieval

Intelligent Systems Approach for Classification and Management of Patients with Headache

Primary headache disorders are the most common complaints worldwide. The socioeconomic and personal impact of headache disorders is enormous, as it is the leading cause of workplace absence. Headache patients’ consultations are increasing as the population has increased in size, live longer and many people have multiple conditions, however, access to specialist services across the UK is currently inequitable because the numbers of trained consultant neurologists in the UK are 10 times lower than other European countries. Additionally, more than two third of headache cases presented to primary care were labelled with unspecified headache. Therefore, an alternative pathway to diagnose and manage patients with primary headache could be crucial to reducing the need for specialist assessment and increase capacity within the current service model. Several recent studies have targeted this issue through the development of clinical decision support systems, which can help non-specialist doctors and general practitioners to diagnose patients with primary headache disorders in primary clinics. However, the majority of these studies were following a rule-based system style, in which the rules were summarised and expressed by a computer engineer. This style carries many downsides, and we will discuss them later on in this dissertation. In this study, we are adopting a completely different approach. The use of machine learning is recruited for the classification of primary headache disorders, for which a dataset of 832 records of patients with primary headaches was considered, originating from three medical centres located in Turkey. Three main types of primary headaches were derived from the data set including Tension Type Headache in both episodic and chronic forms, Migraine with and without Aura, followed by Trigeminal Autonomic Cephalalgia that further subdivided into Cluster headache, paroxysmal hemicrania and short-lasting unilateral neuralgiform headache attacks with conjunctival injection and tearing. Six popular machine-learning based classifiers, including linear and non-linear ensemble learning, in addition to one regression based procedure, have been evaluated for the classification of primary headaches within a supervised learning setting, achieving highest aggregate performance outcomes of AUC 0.923, sensitivity 0.897, and overall classification accuracy of 0.843. This study also introduces the proposed HydroApp system, which is an M-health based personalised application for the follow-up of patients with long-term conditions such as chronic headache and hydrocephalus. We managed to develop this system with the supervision of headache specialists at Ashford hospital, London, and neurology experts at Walton Centre and Alder Hey hospital Liverpool. We have successfully investigated the acceptance of using such an M-health based system via an online questionnaire, where 86% of paediatric patients and 60% of adult patients were interested in using HydroApp system to manage their conditions. Features and functions offered by HydroApp system such as recording headache score, recording of general health and well-being as well as alerting the treating team, have been perceived as very or extremely important aspects from patients’ point of view. The study concludes that the advances in intelligent systems and M-health applications represent a promising atmosphere through which to identify alternative solutions, which in turn increases the capacity in the current service model and improves diagnostic capability in the primary headache domain and beyond