Parsimonious Wavelet Kernel Extreme Learning Machine

In this study, a parsimonious scheme for wavelet kernel extreme learning machine (named PWKELM) was introduced by combining wavelet theory and a parsimonious algorithm into kernel extreme learning machine (KELM). In the wavelet analysis, bases that were localized in time and frequency to represent various signals effectively were used. Wavelet kernel extreme learning machine (WELM) maximized its capability to capture the essential features in “frequency-rich” signals. The proposed parsimonious algorithm also incorporated significant wavelet kernel functions via iteration in virtue of Householder matrix, thus producing a sparse solution that eased the computational burden and improved numerical stability. The experimental results achieved from the synthetic dataset and a gas furnace instance demonstrated that the proposed PWKELM is efficient and feasible in terms of improving generalization accuracy and real time performance

Design and Analysis of A New Illumination Invariant Human Face Recognition System

In this dissertation we propose the design and analysis of a new illumination invariant face recognition system. We show that the multiscale analysis of facial structure and features of face images leads to superior recognition rates for images under varying illumination. We assume that an image I ( x,y ) is a black box consisting of a combination of illumination and reflectance. A new approximation is proposed to enhance the illumination removal phase. As illumination resides in the low-frequency part of images, a high-performance multiresolution transformation is employed to accurately separate the frequency contents of input images. The procedure is followed by a fine-tuning process. After extracting a mask, feature vector is formed and the principal component analysis (PCA) is used for dimensionality reduction which is then proceeded by the extreme learning machine (ELM) as a classifier. We then analyze the effect of the frequency selectivity of subbands of the transformation on the performance of the proposed face recognition system. In fact, we first propose a method to tune the characteristics of a multiresolution transformation, and then analyze how these specifications may affect the recognition rate. In addition, we show that the proposed face recognition system can be further improved in terms of the computational time and accuracy. The motivation for this progress is related to the fact that although illumination mostly lies in the low-frequency part of images, these low-frequency components may have low- or high-resonance nature. Therefore, for the first time, we introduce the resonance based analysis of face images rather than the traditional frequency domain approaches. We found that energy selectivity of the subbands of the resonance based decomposition can lead to superior results with less computational complexity. The method is free of any prior information about the face shape. It is systematic and can be applied separately on each image. Several experiments are performed employing the well known databases such as the Yale B, Extended-Yale B, CMU-PIE, FERET, AT&T, and LFW. Illustrative examples are given and the results confirm the effectiveness of the method compared to the current results in the literature

Automatic analysis and classification of cardiac acoustic signals for long term monitoring

Objective: Cardiovascular diseases are the leading cause of death worldwide resulting in over 17.9 million deaths each year. Most of these diseases are preventable and treatable, but their progression and outcomes are significantly more positive with early-stage diagnosis and proper disease management. Among the approaches available to assist with the task of early-stage diagnosis and management of cardiac conditions, automatic analysis of auscultatory recordings is one of the most promising ones, since it could be particularly suitable for ambulatory/wearable monitoring. Thus, proper investigation of abnormalities present in cardiac acoustic signals can provide vital clinical information to assist long term monitoring. Cardiac acoustic signals, however, are very susceptible to noise and artifacts, and their characteristics vary largely with the recording conditions which makes the analysis challenging. Additionally, there are challenges in the steps used for automatic analysis and classification of cardiac acoustic signals. Broadly, these steps are the segmentation, feature extraction and subsequent classification of recorded signals using selected features. This thesis presents approaches using novel features with the aim to assist the automatic early-stage detection of cardiovascular diseases with improved performance, using cardiac acoustic signals collected in real-world conditions. Methods: Cardiac auscultatory recordings were studied to identify potential features to help in the classification of recordings from subjects with and without cardiac diseases. The diseases considered in this study for the identification of the symptoms and characteristics are the valvular heart diseases due to stenosis and regurgitation, atrial fibrillation, and splitting of fundamental heart sounds leading to additional lub/dub sounds in the systole or diastole interval of a cardiac cycle. The localisation of cardiac sounds of interest was performed using an adaptive wavelet-based filtering in combination with the Shannon energy envelope and prior information of fundamental heart sounds. This is a prerequisite step for the feature extraction and subsequent classification of recordings, leading to a more precise diagnosis. Localised segments of S1 and S2 sounds, and artifacts, were used to extract a set of perceptual and statistical features using wavelet transform, homomorphic filtering, Hilbert transform and mel-scale filtering, which were then fed to train an ensemble classifier to interpret S1 and S2 sounds. Once sound peaks of interest were identified, features extracted from these peaks, together with the features used for the identification of S1 and S2 sounds, were used to develop an algorithm to classify recorded signals. Overall, 99 features were extracted and statistically analysed using neighborhood component analysis (NCA) to identify the features which showed the greatest ability in classifying recordings. Selected features were then fed to train an ensemble classifier to classify abnormal recordings, and hyperparameters were optimized to evaluate the performance of the trained classifier. Thus, a machine learning-based approach for the automatic identification and classification of S1 and S2, and normal and abnormal recordings, in real-world noisy recordings using a novel feature set is presented. The validity of the proposed algorithm was tested using acoustic signals recorded in real-world, non-controlled environments at four auscultation sites (aortic valve, tricuspid valve, mitral valve, and pulmonary valve), from the subjects with and without cardiac diseases; together with recordings from the three large public databases. The performance metrics of the methodology in relation to classification accuracy (CA), sensitivity (SE), precision (P+), and F1 score, were evaluated. Results: This thesis proposes four different algorithms to automatically classify fundamental heart sounds – S1 and S2; normal fundamental sounds and abnormal additional lub/dub sounds recordings; normal and abnormal recordings; and recordings with heart valve disorders, namely the mitral stenosis (MS), mitral regurgitation (MR), mitral valve prolapse (MVP), aortic stenosis (AS) and murmurs, using cardiac acoustic signals. The results obtained from these algorithms were as follows: • The algorithm to classify S1 and S2 sounds achieved an average SE of 91.59% and 89.78%, and F1 score of 90.65% and 89.42%, in classifying S1 and S2, respectively. 87 features were extracted and statistically studied to identify the top 14 features which showed the best capabilities in classifying S1 and S2, and artifacts. The analysis showed that the most relevant features were those extracted using Maximum Overlap Discrete Wavelet Transform (MODWT) and Hilbert transform. • The algorithm to classify normal fundamental heart sounds and abnormal additional lub/dub sounds in the systole or diastole intervals of a cardiac cycle, achieved an average SE of 89.15%, P+ of 89.71%, F1 of 89.41%, and CA of 95.11% using the test dataset from the PASCAL database. The top 10 features that achieved the highest weights in classifying these recordings were also identified. • Normal and abnormal classification of recordings using the proposed algorithm achieved a mean CA of 94.172%, and SE of 92.38%, in classifying recordings from the different databases. Among the top 10 acoustic features identified, the deterministic energy of the sound peaks of interest and the instantaneous frequency extracted using the Hilbert Huang-transform, achieved the highest weights. • The machine learning-based approach proposed to classify recordings of heart valve disorders (AS, MS, MR, and MVP) achieved an average CA of 98.26% and SE of 95.83%. 99 acoustic features were extracted and their abilities to differentiate these abnormalities were examined using weights obtained from the neighborhood component analysis (NCA). The top 10 features which showed the greatest abilities in classifying these abnormalities using recordings from the different databases were also identified. The achieved results demonstrate the ability of the algorithms to automatically identify and classify cardiac sounds. This work provides the basis for measurements of many useful clinical attributes of cardiac acoustic signals and can potentially help in monitoring the overall cardiac health for longer duration. The work presented in this thesis is the first-of-its-kind to validate the results using both, normal and pathological cardiac acoustic signals, recorded for a long continuous duration of 5 minutes at four different auscultation sites in non-controlled real-world conditions.Open Acces

Metodología para la selección automática de características de señales EEG utilizando algoritmos de aprendizaje de máquina aplicado al reconocimiento del procesamiento emocional en excombatientes

Los diferentes procesos de desmovilización y reincorporación a la vida en sociedad han conseguido que miles de excombatientes de diferentes grupos armados busquen retornar a la vida civil. Sin embargo, se ha reportado en la literatura que la experiencia de guerra causa en estas personas trastornos y desordenes psicológicos que les impiden completar su proceso de reintegración. Se ha encontrado en la literatura distintas alternativas para estudiar el comportamiento de personas que han participado en el conflicto armado, algunos de estos métodos abordan el problema desde la psicología, haciendo entrevistas y encuestas asistidas por expertos. En los últimos años estos estudios han sido apoyados cada vez más con técnicas de aprendizaje de máquina, haciendo análisis de registros electroencefalográficos (EEG), ya que el uso de los sensores para la adquisición de estas señales tiene un costo reducido y la prueba es no invasiva, lo cual facilita poner en práctica esta técnica. Además, los registros EEG tienen una muy buena resolución temporal (milisegundos), y mediante su análisis se ha mostrado una mejoría considerable en el rendimiento de la tarea de clasificación entre las clases (controles y excombatientes). La metodología desarrollada fue probada en dos bases de datos que evalúan el procesamiento emocional de controles y sujetos expuesto al conflicto. El primer conjunto de datos tiene como objetivo discriminar entre las clases utilizando una tarea de valencia contextual, y el segundo utiliza estímulos con imágenes anqueadas para distinguir entre sujetos que han recibido alta exposición y sujetos con baja exposición al conflicto armado colombiano. En esta tesis se plantea una metodología que utiliza dos formas de caracterización de registros EEG utilizando combinación de la representación de estas señales en tiempo, frecuencia y espacio. El primero de los métodos utiliza caracterización en tiempo-frecuencia empleando la transformada Wavelet en su forma discreta para descomponer las señales EEG. Después se extrajeron datos estadísticos sobre los coeficientes de detalle y aproximación, los cuales fueron utilizados como características. Por otro lado, se utilizó también información en frecuencia-espacio, haciendo análisis de conectividad funcional y aplicando la teoría de grafos a las conexiones encontradas en diferentes escalas de conectividad. Adicionalmente, se realizó un análisis de relevancia con tres métodos que permiten brindar mayor interpretabilidad a los resultados obtenidos y obtener una mayor tasa de clasificación al utilizar las características más relevantes. Los métodos utilizados son búsqueda exhaustiva, aprendizaje multi kernel (MKL), y selección de características con ANOVA. Finalmente, se realiza la clasificación de las características con una máquina de vectores de soporte, obteniendo el puntaje F1 como medida de evaluación. Los resultados sugieren que existe diferencia entre las clases de la tarea denominada como Flanker, consiguiendo hasta 94% de puntaje F1 en la tarea de clasificación. Para el caso de valencia contextual se tiene hasta un 85% en el puntaje F1 combinando la información espectral con MKL. En general, se obtuvo que el análisis por bandas de frecuencia obtiene a lo largo de las pruebas los resultados m as altos, aunque el análisis de relevancia con MKL es también consistente, y se observó que la banda en donde se dieron los mejores resultados fue en los rangos de frecuencia altos de B. Esto sugiere que los controles y pacientes expuestos al conflicto presentan una diferencia en los niveles de concentración y atención.The different demobilization process have bring thousands of colombian excombatants in searching for return to the civil life. However, it is reported that the war experience produces psychological disorders that prevent completing their reintegration process. The literature shows several alternatives to study the behavior of people with war experiences, some of these methods address the problem using psychology, i.e., making interviews by experts. In the last years, the studies have been helped by arti cial intelligence using electroencephalographic (EEG) signals, due to EEG is a non-invasive and low-cost study, which facilitates put into practice this technique. Also, EEG signals have an adequate time resolution (milliseconds), and with its analysis the classi cation task between excombatants and controls have improved. The developed methodology was evaluated in two di erent datasets, both assess the emotion processing in controls and subjects with high exposure to the armed con ict. The rst dataset aims to discriminate between classes using contextual valence. The second dataset uses stimuli with anker images to distinguish between subjects that have been highly exposed to the con ict and subjects with low exposure. In this thesis it is developed a methodology that uses two ways of EEG characterization making combinations of the representations of these signals in di erent domains as time, frequency, and space. The rst approach uses features in time-frequency domain employing decomposition with multiple discrete wavelets, then, statistics features are extracted from the decomposition coe cients. On the other hand, it is used frequency-space information making a functional connectivity analysis and applying graph theory over the connections found on the connectivity. Also, it was made a feature relevance analysis through three methods that give better interpretability of the data. The relevance analysis methods used are: Exhaustive search with frequency bands, weights assignment with MKL, and ANOVA feature selection technique. Finally, the classi cation was made using SVM, and evaluated with the F1 score metric. Results suggest that there is a di erence between the classes in the anker dataset, reaching a 94% of F1 score. For the contextual valence dataset, the F1 score achieves an 85% by combining the spectral information with MKL. In general the exhaustive search method showed the best scores among several tests, nevertheless the relevance analysis with MKL is the most regular method. Finally, it is shown that higher frequencies in the beta band are the most relevant ones, suggesting that the controls and subjects present di erences in the concentration and attention level

Ensemble approach on enhanced compressed noise EEG data signal in wireless body area sensor network

The Wireless Body Area Sensor Network (WBASN) is used for communication among sensor nodes operating on or inside the human body in order to monitor vital body parameters and movements. One of the important applications of WBASN is patients’ healthcare monitoring of chronic diseases such as epileptic seizure. Normally, epileptic seizure data of the electroencephalograph (EEG) is captured and compressed in order to reduce its transmission time. However, at the same time, this contaminates the overall data and lowers classification accuracy. The current work also did not take into consideration that large size of collected EEG data. Consequently, EEG data is a bandwidth intensive. Hence, the main goal of this work is to design a unified compression and classification framework for delivery of EEG data in order to address its large size issue. EEG data is compressed in order to reduce its transmission time. However, at the same time, noise at the receiver side contaminates the overall data and lowers classification accuracy. Another goal is to reconstruct the compressed data and then recognize it. Therefore, a Noise Signal Combination (NSC) technique is proposed for the compression of the transmitted EEG data and enhancement of its classification accuracy at the receiving side in the presence of noise and incomplete data. The proposed framework combines compressive sensing and discrete cosine transform (DCT) in order to reduce the size of transmission data. Moreover, Gaussian noise model of the transmission channel is practically implemented to the framework. At the receiving side, the proposed NSC is designed based on weighted voting using four classification techniques. The accuracy of these techniques namely Artificial Neural Network, Naïve Bayes, k-Nearest Neighbour, and Support Victor Machine classifiers is fed to the proposed NSC. The experimental results showed that the proposed technique exceeds the conventional techniques by achieving the highest accuracy for noiseless and noisy data. Furthermore, the framework performs a significant role in reducing the size of data and classifying both noisy and noiseless data. The key contributions are the unified framework and proposed NSC, which improved accuracy of the noiseless and noisy EGG large data. The results have demonstrated the effectiveness of the proposed framework and provided several credible benefits including simplicity, and accuracy enhancement. Finally, the research improves clinical information about patients who not only suffer from epilepsy, but also neurological disorders, mental or physiological problems

Image receptive fields for artificial neural networks

This paper describes the structure of the Image Receptive Fields Neural Network (IRF-NN) introduced recently by our team. This structure extends simplified learning introduced by Extreme Learning Machine and Reservoir Computing techniques to the field of images. Neurons are organized in a single hidden layer feedforward network architecture with an original organization of the network׳s input weights. To represent color images efficiently, without prior feature extraction, the weight values linked to a neuron are determined by a 2-D Gaussian function. The activation of a neuron by an image presents the properties of a nonlinear localized receptive field, parameterized with a small number of degrees of freedom. A network composed of a large number of neurons, each associated with a randomly initialized and constant receptive field, induces a remarkable representation of the images. Supervised training determines only the output weights of the network. It is therefore extremely fast, without retropropagation or iterations, adapted to large sets of images. The network is easy to implement, presents excellent generalization performances for classification applications, and allows the detection of unknown inputs. The efficiency of this technique is illustrated with several benchmarks, photo and video datasets

Heart Diseases Diagnosis Using Artificial Neural Networks

Information technology has virtually altered every aspect of human life in the present era. The application of informatics in the health sector is rapidly gaining prominence and the benefits of this innovative paradigm are being realized across the globe. This evolution produced large number of patients’ data that can be employed by computer technologies and machine learning techniques, and turned into useful information and knowledge. This data can be used to develop expert systems to help in diagnosing some life-threating diseases such as heart diseases, with less cost, processing time and improved diagnosis accuracy. Even though, modern medicine is generating huge amount of data every day, little has been done to use this available data to solve challenges faced in the successful diagnosis of heart diseases. Highlighting the need for more research into the usage of robust data mining techniques to help health care professionals in the diagnosis of heart diseases and other debilitating disease conditions. Based on the foregoing, this thesis aims to develop a health informatics system for the classification of heart diseases using data mining techniques focusing on Radial Basis functions and emerging Neural Networks approach. The presented research involves three development stages; firstly, the development of a preliminary classification system for Coronary Artery Disease (CAD) using Radial Basis Function (RBF) neural networks. The research then deploys the deep learning approach to detect three different types of heart diseases i.e. Sleep Apnea, Arrhythmias and CAD by designing two novel classification systems; the first adopt a novel deep neural network method (with Rectified Linear unit activation) design as the second approach in this thesis and the other implements a novel multilayer kernel machine to mimic the behaviour of deep learning as the third approach. Additionally, this thesis uses a dataset obtained from patients, and employs normalization and feature extraction means to explore it in a unique way that facilitates its usage for training and validating different classification methods. This unique dataset is useful to researchers and practitioners working in heart disease treatment and diagnosis. The findings from the study reveal that the proposed models have high classification performance that is comparable, or perhaps exceed in some cases, the existing automated and manual methods of heart disease diagnosis. Besides, the proposed deep-learning models provide better performance when applied on large data sets (e.g., in the case of Sleep Apnea), with reasonable performance with smaller data sets. The proposed system for clinical diagnoses of heart diseases, contributes to the accurate detection of such disease, and could serve as an important tool in the area of clinic support system. The outcome of this study in form of implementation tool can be used by cardiologists to help them make more consistent diagnosis of heart diseases

Meta-learning for Forecasting Model Selection

Model selection for time series forecasting is a challenging task for practitioners and academia. There are multiple approaches to address this, ranging from time series analysis using a series of statistical tests, to information criteria or empirical approaches that rely on cross-validated errors. In recent forecasting competitions, meta-learning obtained promising results establishing its place as a model selection alternative. Meta-learning constructs meta-features for each time series and trains a classifier on these to choose the most appropriate forecasting method. In the first part, this thesis studies the main components of meta-learning and analyses the effect of alternative meta-features, meta-learners, and base forecasters in the final model selection results. We investigate different meta-learners, the use of simple or complex base forecasts, and a large and diverse set of meta-features. Our findings show that stationarity tests, which identify the presence of unit root in time series, and proxies of autoregressive information, which show the strength of serial correlation in a series, have the highest importance for the performance of meta-learning. On the contrary, features related to time series quantiles and other descriptive statistics such as the mean, and the variance exhibit the lowest importance. Furthermore, we observe that using simple base forecasters is more sensitive to the number of groups of features employed as meta-feature and overall had worse performed. In terms of the choice of learners, classifiers with evidence of good performance in the literature resulted in the most accurate meta-learners. The success of meta-learning largely depends on its building components. The selection and generation of the appropriate meta-features remains a major challenge in meta-learning. In the second part, we propose using Convolutional Neural Networks (CNN) to overcome this. CNN have demonstrated breakthrough accuracy in pattern recognition tasks and can generate features as needed internally, within its layers, without intervention from the modeller. Using CNN, we provide empirical evidence of the efficacy of the approach, against widely accepted forecast selection methods and discuss the advantages and limitations of the proposed approach. Finally, we provide additional evidence that using meta-learning, for automated model selection, outperformed all of the individual benchmark forecasts