Multi-class SVMs: From Tighter Data-Dependent Generalization Bounds to Novel Algorithms

This paper studies the generalization performance of multi-class classification algorithms, for which we obtain, for the first time, a data-dependent generalization error bound with a logarithmic dependence on the class size, substantially improving the state-of-the-art linear dependence in the existing data-dependent generalization analysis. The theoretical analysis motivates us to introduce a new multi-class classification machine based on $\ell_p$-norm regularization, where the parameter $p$ controls the complexity of the corresponding bounds. We derive an efficient optimization algorithm based on Fenchel duality theory. Benchmarks on several real-world datasets show that the proposed algorithm can achieve significant accuracy gains over the state of the art

Novel gumbel-softmax trick enabled concrete autoencoder with entropy constraints for unsupervised hyperspectral band selection.

As an important topic in hyperspectral image (HSI) analysis, band selection has attracted increasing attention in the last two decades for dimensionality reduction in HSI. With the great success of deep learning (DL)-based models recently, a robust unsupervised band selection (UBS) neural network is highly desired, particularly due to the lack of sufficient ground truth information to train the DL networks. Existing DL models for band selection either depend on the class label information or have unstable results via ranking the learned weights. To tackle these challenging issues, in this article, we propose a Gumbel-Softmax (GS) trick enabled concrete autoencoder-based UBS framework (CAE-UBS) for HSI, in which the learning process is featured by the introduced concrete random variables and the reconstruction loss. By searching from the generated potential band selection candidates from the concrete encoder, the optimal band subset can be selected based on an information entropy (IE) criterion. The idea of the CAE-UBS is quite straightforward, which does not rely on any complicated strategies or metrics. The robust performance on four publicly available datasets has validated the superiority of our CAE-UBS framework in the classification of the HSIs

Effective EEG analysis for advanced AI-driven motor imagery BCI systems

Developing effective signal processing for brain-computer interfaces (BCIs) and brain-machine interfaces (BMIs) involves factoring in three aspects of functionality: classification performance, execution time, and the number of data channels used. The contributions in this thesis are centered on these three issues. Contributions are focused on the classification of motor imagery (MI) data, which is generated during imagined movements. Typically, EEG time-series data is segmented for data augmentation or to mimic buffering that happens in an online BCI. A multi-segment decision fusion approach is presented, which takes consecutive temporal segments of EEG data, and uses decision fusion to boost classification performance. It was computationally lightweight and improved the performance of four conventional classifiers. Also, an analysis of the contributions of electrodes from different scalp regions is presented, and a subset of channels is recommended. Sparse learning (SL) classifiers have exhibited strong classification performance in the literature. However, they are computationally expensive. To reduce the test-set execution times, a novel EEG classification pipeline consisting of a genetic-algorithm (GA) for channel selection and a dictionary-based SL module for classification, called GABSLEEG, is presented. Subject-specific channel selection was carried out, in which the channels are selected based on training data from the subject. Using the GA-recommended subset of EEG channels reduced the execution time by 60% whilst preserving classification performance. Although subject-specific channel selection is widely used in the literature, effective subject-independent channel selection, in which channels are detected using data from other subjects, is an ideal aim because it leads to lower training latency and reduces the number of electrodes needed. A novel convolutional neural network (CNN)-based subject-independent channels selection method is presented, called the integrated channel selection (ICS) layer. It performed on-a-par with or better than subject-specific channel selection. It was computationally efficient, operating 12-17 times faster than the GA channel selection module. The ICS layer method was versatile, performing well with two different CNN architectures and datasets.Developing effective signal processing for brain-computer interfaces (BCIs) and brain-machine interfaces (BMIs) involves factoring in three aspects of functionality: classification performance, execution time, and the number of data channels used. The contributions in this thesis are centered on these three issues. Contributions are focused on the classification of motor imagery (MI) data, which is generated during imagined movements. Typically, EEG time-series data is segmented for data augmentation or to mimic buffering that happens in an online BCI. A multi-segment decision fusion approach is presented, which takes consecutive temporal segments of EEG data, and uses decision fusion to boost classification performance. It was computationally lightweight and improved the performance of four conventional classifiers. Also, an analysis of the contributions of electrodes from different scalp regions is presented, and a subset of channels is recommended. Sparse learning (SL) classifiers have exhibited strong classification performance in the literature. However, they are computationally expensive. To reduce the test-set execution times, a novel EEG classification pipeline consisting of a genetic-algorithm (GA) for channel selection and a dictionary-based SL module for classification, called GABSLEEG, is presented. Subject-specific channel selection was carried out, in which the channels are selected based on training data from the subject. Using the GA-recommended subset of EEG channels reduced the execution time by 60% whilst preserving classification performance. Although subject-specific channel selection is widely used in the literature, effective subject-independent channel selection, in which channels are detected using data from other subjects, is an ideal aim because it leads to lower training latency and reduces the number of electrodes needed. A novel convolutional neural network (CNN)-based subject-independent channels selection method is presented, called the integrated channel selection (ICS) layer. It performed on-a-par with or better than subject-specific channel selection. It was computationally efficient, operating 12-17 times faster than the GA channel selection module. The ICS layer method was versatile, performing well with two different CNN architectures and datasets

Recognizing complex faces and gaits via novel probabilistic models

In the field of computer vision, developing automated systems to recognize people under unconstrained scenarios is a partially solved problem. In unconstrained sce- narios a number of common variations and complexities such as occlusion, illumi- nation, cluttered background and so on impose vast uncertainty to the recognition process. Among the various biometrics that have been emerging recently, this dissertation focus on two of them namely face and gait recognition. Firstly we address the problem of recognizing faces with major occlusions amidst other variations such as pose, scale, expression and illumination using a novel PRObabilistic Component based Interpretation Model (PROCIM) inspired by key psychophysical principles that are closely related to reasoning under uncertainty. The model basically employs Bayesian Networks to establish, learn, interpret and exploit intrinsic similarity mappings from the face domain. Then, by incorporating e cient inference strategies, robust decisions are made for successfully recognizing faces under uncertainty. PROCIM reports improved recognition rates over recent approaches. Secondly we address the newly upcoming gait recognition problem and show that PROCIM can be easily adapted to the gait domain as well. We scienti cally de ne and formulate sub-gaits and propose a novel modular training scheme to e ciently learn subtle sub-gait characteristics from the gait domain. Our results show that the proposed model is robust to several uncertainties and yields sig- ni cant recognition performance. Apart from PROCIM, nally we show how a simple component based gait reasoning can be coherently modeled using the re- cently prominent Markov Logic Networks (MLNs) by intuitively fusing imaging, logic and graphs. We have discovered that face and gait domains exhibit interesting similarity map- pings between object entities and their components. We have proposed intuitive probabilistic methods to model these mappings to perform recognition under vari- ous uncertainty elements. Extensive experimental validations justi es the robust- ness of the proposed methods over the state-of-the-art techniques.

New Fundamental Technologies in Data Mining

The progress of data mining technology and large public popularity establish a need for a comprehensive text on the subject. The series of books entitled by "Data Mining" address the need by presenting in-depth description of novel mining algorithms and many useful applications. In addition to understanding each section deeply, the two books present useful hints and strategies to solving problems in the following chapters. The contributing authors have highlighted many future research directions that will foster multi-disciplinary collaborations and hence will lead to significant development in the field of data mining

Computational investigation of systemic pathway responses in severe pneumonia among the Gambian children and infants

Pneumonia remains the leading cause of infectious mortality in under-five children, and the burden is highest in sub-Saharan Africa. To mitigate this burden, further knowledge is required to accelerate the development of innovative and cost-effective approaches. To gain a deeper insight into the pathogenesis of pneumonia, I investigated the central hypothesis that systemic pathway (cellular and molecular) responses underpin the development of severe pneumonia outcomes. Mainly, I compared whole blood transcriptomes between severe pneumonia cases (clinically stratified as mild, severe and very severe) and non-pneumonia community controls (prospectively matched by age and sex). In total, 803 whole blood RNA samples were collected from Gambian children (aged 2-59 months) between 2007 and 2010, of which, 518 passed laboratory quality control criteria for the microarray analysis. After data cleaning, the final database reduced to 503 samples including the training (n=345) and independent validation (n=158) data sets. To investigate the cellular responses, I applied computational deconvolution analysis to assess the variations of immune cell type proportions with pneumonia severity. To further enhance the computational performance, I applied a data fusion approach on 3,475 immune marker genes from different resources to derive an optimal and integrated blood marker list (IBML, m=277) for Neutrophils, Monocytes, NK, Dendritic, B and T cell types; which robustly performed better than the existing individual resources. Using the IBML resource, pneumonia severity was significantly associated with the depletion of B, T, Dendritic and NK cell types, and the elevation of Monocytes and neutrophil proportions (P-value<0.001). At the molecular level, pneumonia severity was associated (false discovery rate<0.05) with a battery of systemic pathway (innate, adaptive and metabolic) responses in a range of biomedical databases. While the up-regulation of inflammatory innate responses was also observed in mild cases, severe pneumonia cases were predominantly associated with the co-inhibition of the cells of the adaptive immune response (B and T) and Natural killer cells, and the up-regulation of fatty acid and lipid metabolism. While most of these findings were anticipated, the involvement of NK cells was unexpected, and potentially presents a novel immune-modulation target for mitigating the burden of pneumonia. Together, the cellular and molecular pathways responses consistently support the central hypothesis that systemic pathway responses contribute significantly to the development of severe pneumonia outcomes. Clinically, the identification and appropriate treatment of patients at the higher risk of developing severe pneumonia outcomes remains the major challenge. To address that, I applied supervised machine-learning approaches on cellular pathway based transcriptomic features; and derived a 33-gene classifier (representing the NK, T, and neutrophils cell types), which accurately detected severe pneumonia cases in both the training (leave-one-out cross-validated accuracy=99%) and independent validation (accuracy=98%) datasets. Independently, similar performance (98% in each dataset) was associated with a subset (m=18) of the validated 52-gene neonatal sepsis classifier. Conversely, at least 75% of the cellular biomarkers were differentially expressed (false discovery rate<0.05) in bacterial neonatal sepsis. Further, very severe pneumonia cases were predominantly associated with antibacterial responses; and mild pneumonia cases with blood-culture-confirmed positivity were also associated with an increased frequency of differentially expressed genes. These findings suggest the significant contribution of bacterial septicaemia in the development of serious pneumonia outcomes. Together, this study highlights the future potential of host-derived systemic biomarkers for early identification and novel treatment modalities of high-risk cases presenting at a resource-constrained clinic with mild pneumonia. However, further validation studies are required

Human perception capabilities for socially intelligent domestic service robots

The daily living activities for an increasing number of frail elderly people represent a continuous struggle both for them as well as for their extended families. These people have difficulties coping at home alone but are still sufficiently fit not to need the round-the-clock care provided by a nursing home. Their struggle can be alleviated by the deployment of a mechanical helper in their home, i.e. a service robot that can execute a range of simple object manipulation tasks. Such a robotic application promises to extend the period of independent home living for elderly people, while providing them with a better quality of life. However, despite the recent technological advances in robotics, there are still some remaining challenges, mainly related to the human factors. Arguably, the lack of consistently dependable human detection, localisation, position and pose tracking information and insufficiently refined processing of sensor information makes the close range physical interaction between a robot and a human a high-risk task. The work described in this thesis addresses the deficiencies in the processing of the human information of today’s service robots. This is achieved through proposing a new paradigm for the robot’s situational awareness in regard to people as well as a collection of methods and techniques, operating at the lower levels of the paradigm, i.e. perception of new human information. The collection includes methods for obtaining and processing of information about the presence, location and body pose of the people. In addition to the availability of reliable human perception information, the integration between the separate levels of paradigm is considered to be a critically important factor for achieving the human-aware control of the robot. Improving the cognition, judgment and decision making action links between the paradigm’s layers leads to enhanced capability of the robot to engage in a natural and more meaningful interaction with people and, therefore, to a more enjoyable user experience. Therefore, the proposed paradigm and methodology are envisioned to contribute to making the prolonged assisted living of elderly people at home a more feasible and realistic task. In particular, this thesis proposes a set of methods for human presence detection, localisation and body pose tracking that are operating on the perception level of the paradigm. Also, the problem of having only limited visibility of a person from the on-board sensors of the robot is addressed by the proposed classifier fusion method that combines information from several types of sensors. A method for improved real-time human body pose tracking is also investigated. Additionally, a method for estimation of the multiple human tracks from noisy detections, as well as analysis of the computed human tracks for cognition about the social interactions within the social group, operating at the comprehension level of the robot’s situational awareness paradigm, is proposed. Finally, at the human-aware planning layer, a method that utilises the human related information, generated by the perception and comprehension layers to compute a minimally intrusive navigation path to a target person within a human group, is proposed. This method demonstrates how the improved human perception capabilities of the robot, through its judgement activity, ii ABSTRACT can be utilised by the highest level of the paradigm, i.e. the decision making layer, to achieve user friendly human-robot interactions. Overall, the research presented in this work, drawing on recent innovation in statistical learning, data fusion and optimisation methods, improves the overall situational awareness of the robot in regard to people with the main focus placed on human sensing capabilities of service robots. The improved overall situational awareness of the robot regarding people, as defined by the proposed paradigm, enables more meaningful human-robot interactions