Data-driven multivariate and multiscale methods for brain computer interface

This thesis focuses on the development of data-driven multivariate and multiscale methods for brain computer interface (BCI) systems. The electroencephalogram (EEG), the most convenient means to measure neurophysiological activity due to its noninvasive nature, is mainly considered. The nonlinearity and nonstationarity inherent in EEG and its multichannel recording nature require a new set of data-driven multivariate techniques to estimate more accurately features for enhanced BCI operation. Also, a long term goal is to enable an alternative EEG recording strategy for achieving long-term and portable monitoring. Empirical mode decomposition (EMD) and local mean decomposition (LMD), fully data-driven adaptive tools, are considered to decompose the nonlinear and nonstationary EEG signal into a set of components which are highly localised in time and frequency. It is shown that the complex and multivariate extensions of EMD, which can exploit common oscillatory modes within multivariate (multichannel) data, can be used to accurately estimate and compare the amplitude and phase information among multiple sources, a key for the feature extraction of BCI system. A complex extension of local mean decomposition is also introduced and its operation is illustrated on two channel neuronal spike streams. Common spatial pattern (CSP), a standard feature extraction technique for BCI application, is also extended to complex domain using the augmented complex statistics. Depending on the circularity/noncircularity of a complex signal, one of the complex CSP algorithms can be chosen to produce the best classification performance between two different EEG classes. Using these complex and multivariate algorithms, two cognitive brain studies are investigated for more natural and intuitive design of advanced BCI systems. Firstly, a Yarbus-style auditory selective attention experiment is introduced to measure the user attention to a sound source among a mixture of sound stimuli, which is aimed at improving the usefulness of hearing instruments such as hearing aid. Secondly, emotion experiments elicited by taste and taste recall are examined to determine the pleasure and displeasure of a food for the implementation of affective computing. The separation between two emotional responses is examined using real and complex-valued common spatial pattern methods. Finally, we introduce a novel approach to brain monitoring based on EEG recordings from within the ear canal, embedded on a custom made hearing aid earplug. The new platform promises the possibility of both short- and long-term continuous use for standard brain monitoring and interfacing applications

時間周波数領域でのてんかん脳波識別に関する研究 ‐平均二乗根に基づく特徴抽出に着目して‐

Epilepsy affects over 50 million people on an average yearly world wide. Epileptic Seizure is a generalised term which has broad classification depending on the reasons behind its occurrence. Parvez et al. when applied feature instantaneous bandwidth B2AM and time averaged bandwidth B2FM for classification of interictal and ictal on Freiburg data base, the result dipped low to 77.90% for frontal lobe whereas it was 80.20% for temporal lobe compare to the 98.50% of classification accuracy achieved on Bonn dataset with same feature for classification of ictal against interictal. We found reasons behind such low results are, first Parvez et al. has used first IMF of EMD for feature computation which mostly noised induce. Secondly, they used same kernel parameters of SVM as Bajaj et al. which they must have optimised with different dataset. But the most important reason we found is that two signals s1 and s2 can have same instantaneous bandwidth. Therefore, the motivation of the dissertation is to address the drawback of feature instantaneous bandwidth by new feature with objective of achieving comparable classification accuracy. In this work, we have classified ictal from healthy nonseizure interictal successfully first by using RMS frequency and another feature from Hilbert marginal spectrum then with its parameters ratio. RMS frequency is the square root of sum of square bandwidth and square of center frequency. Its contributing parameters ratio is ratio of center frequency square to square bandwidth. We have also used dominant frequency and its parameters ratio for the same purpose. Dominant frequency have same physical relevance as RMS frequency but different by definition, i.e. square root of sum of square of instantaneous band- width and square of instantaneous frequency. Third feature that we have used is by exploiting the equivalence of RMS frequency and dominant frequency (DF) to define root mean instantaneous frequency square (RMIFS) as square root of sum of time averaged bandwidth square and center frequency square. These features are average measures which shows good discrimination power in classifying ictal from interictal using SVM. These features, fr and fd also have an advantage of overcoming the draw back of square bandwidth and instantaneous bandwidth. RMS frequency that we have used in this work is different from generic root mean square analysis. We have used an adaptive thresholding algorithm to address the issue of false positive. It was able to increase the specificity by average of 5.9% on average consequently increasing the accuracy. Then we have applied morphological component analysis (MCA) with the fractional contribution of dominant frequency and other rest of the features like band- width parameter’s contribution and RMIFS frequency and its parameters and their ratio. With the results from proposed features, we validated our claim to overcome the drawback of instantaneous bandwidth and square bandwidth.九州工業大学博士学位論文 学位記番号：生工博甲第323号 学位授与年月日：平成30年6月28日1 Introduction|2 Empirical Mode Decomposition|3 Root Mean Square Frequency|4 Root Mean Instantaneous Frequency Square|5 Morphological Component Analysis|6 Conclusion九州工業大学平成30年

Electroencephalographic Signal Processing and Classification Techniques for Noninvasive Motor Imagery Based Brain Computer Interface

In motor imagery (MI) based brain-computer interface (BCI), success depends on reliable processing of the noisy, non-linear, and non-stationary brain activity signals for extraction of features and effective classification of MI activity as well as translation to the corresponding intended actions. In this study, signal processing and classification techniques are presented for electroencephalogram (EEG) signals for motor imagery based brain-computer interface. EEG signals have been acquired placing the electrodes following the international 10-20 system. The acquired signals have been pre-processed removing artifacts using empirical mode decomposition (EMD) and two extended versions of EMD, ensemble empirical mode decomposition (EEMD), and multivariate empirical mode decomposition (MEMD) leading to better signal to noise ratio (SNR) and reduced mean square error (MSE) compared to independent component analysis (ICA). EEG signals have been decomposed into independent mode function (IMFs) that are further processed to extract features like sample entropy (SampEn) and band power (BP). The extracted features have been used in support vector machines to characterize and identify MI activities. EMD and its variants, EEMD, MEMD have been compared with common spatial pattern (CSP) for different MI activities. SNR values from EMD, EEMD and MEMD (4.3, 7.64, 10.62) are much better than ICA (2.1) but accuracy of MI activity identification is slightly better for ICA than EMD using BP and SampEn. Further work is outlined to include more features with larger database for better classification accuracy

Analysis of Signal Decomposition and Stain Separation methods for biomedical applications

Nowadays, the biomedical signal processing and classification and medical image interpretation play an essential role in the detection and diagnosis of several human diseases. The problem of high variability and heterogeneity of information, which is extracted from digital data, can be addressed with signal decomposition and stain separation techniques which can be useful approaches to highlight hidden patterns or rhythms in biological signals and specific cellular structures in histological color images, respectively. This thesis work can be divided into two macro-sections. In the first part (Part I), a novel cascaded RNN model based on long short-term memory (LSTM) blocks is presented with the aim to classify sleep stages automatically. A general workflow based on single-channel EEG signals is developed to enhance the low performance in staging N1 sleep without reducing the performances in the other sleep stages (i.e. Wake, N2, N3 and REM). In the same context, several signal decomposition techniques and time-frequency representations are deployed for the analysis of EEG signals. All extracted features are analyzed by using a novel correlation-based timestep feature selection and finally the selected features are fed to a bidirectional RNN model. In the second part (Part II), a fully automated method named SCAN (Stain Color Adaptive Normalization) is proposed for the separation and normalization of staining in digital pathology. This normalization system allows to standardize digitally, automatically and in a few seconds, the color intensity of a tissue slide with respect to that of a target image, in order to improve the pathologist’s diagnosis and increase the accuracy of computer-assisted diagnosis (CAD) systems. Multiscale evaluation and multi-tissue comparison are performed for assessing the robustness of the proposed method. In addition, a stain normalization based on a novel mathematical technique, named ICD (Inverse Color Deconvolution) is developed for immunohistochemical (IHC) staining in histopathological images. In conclusion, the proposed techniques achieve satisfactory results compared to state-of-the-art methods in the same research field. The workflow proposed in this thesis work and the developed algorithms can be employed for the analysis and interpretation of other biomedical signals and for digital medical image analysis

Altered EEG Oscillatory Brain Networks During Music-Listening in Major Depression

To examine the electrophysiological underpinnings of the functional networks involved in music listening, previous approaches based on spatial independent component analysis (ICA) have recently been used to ongoing electroencephalography (EEG) and magnetoencephalography (MEG). However, those studies focused on healthy subjects, and failed to examine the group-level comparisons during music listening. Here, we combined group-level spatial Fourier ICA with acoustic feature extraction, to enable group comparisons in frequency-specific brain networks of musical feature processing. It was then applied to healthy subjects and subjects with major depressive disorder (MDD). The music-induced oscillatory brain patterns were determined by permutation correlation analysis between individual time courses of Fourier-ICA components and musical features. We found that (1) three components, including a beta sensorimotor network, a beta auditory network and an alpha medial visual network, were involved in music processing among most healthy subjects; and that (2) one alpha lateral component located in the left angular gyrus was engaged in music perception in most individuals with MDD. The proposed method allowed the statistical group comparison, and we found that: (1) the alpha lateral component was activated more strongly in healthy subjects than in the MDD individuals, and that (2) the derived frequency-dependent networks of musical feature processing seemed to be altered in MDD participants compared to healthy subjects. The proposed pipeline appears to be valuable for studying disrupted brain oscillations in psychiatric disorders during naturalistic paradigms.Peer reviewe

Diagnosis of low-speed bearings via vibration-based entropy indicators and acoustic emissions

Tesi del Pla de doctorat industrial de la Generalitat de Catalunya. Tesi en modalitat compendi de publicacions, amb diferents seccions retallades per drets dels editorsWind energy is one ofthe main renewable energies to replace fossil fuels in the generation of electricityworldwide. To enhance and accelerate its implementation at a large scale, it is vital to reduce the costs associated with maintenance. As com ponent breakages force the turbine to stop for long repair times, the wind industry m ust switch from the old-fashioned preventive or corrective maintenance to condition-based maintenance (also called predictive maintenance). The condition­based maintenance of pitch bearings is especiallychallenging, as the operating conditions include high mechanical stress and low rotational speed. Since these operating conditions im pact negatively on the results of the standard methods and techniques applied in current condition-based monitoring systems, the condition-based maintenance of pitch bearings is still a challenge. Therefore, this thes is is focused on the research of novel methods and techniques that obtain reliable information on the state of pitch bearings for condition-based maintenance. lnitially, the acknowledgment ofthe state ofthe art is performed to recognize the methods and signals. This step endorses the decision to analyze the vibration signals and acoustic emissions throughout this thesis. Due to the particular operating conditions of pitch bearings, this research states the need to create data sets to replicate the particular operating conditions in a controlled laboratory experiment. As a res ult, a datas et based on vibrations, and a second datas et based on acoustic emissions are generated. The vibration datas et allows the validation of a novel algorithm for the low-speed bearing diagnosis, which is based on the concept of entropy by the definition of Shannon and Rényi. In com parison to the classical methods found in the literature, the diagnosis of low-speed bearings based on entropy-based indicators can extract more reliable information. Moreover, the research of the com bination of several indicators to improve the diagnosis revea Is that the entropy-based indicators can extract more information than regular indicators used in academia. The datas et of acoustic emissions from low-speed bearings helps to contribute to the development of methods for diagnosis. In this research, the analysis of the energyfrom the signals reveals a dependencyon the intensityand the presence of damage. In addition, a relation between the waveform ofthe analyzed energy and the existence of damage is em phas ized.La energía eólica es una de las principales energías renovables consideradas para reemplazar los combustibles fósiles en la generación de electricidad a nivel mundial. Para mejorar y acelerar su implementación a gran escala, es vital reducir los costes asociados con el mantenimiento. Como las roturas de los componentes obligan a la turbina a detenerse durante largos períodos de reparación, la industria eólica necesita cambiar del anticuado mantenimiento preventiv o correctivo al mantenimiento basado en la condición (también llamado mantenimiento predictivo). El mantenimiento basado en la condición de los rodamientos pitch es especialmente desafiante, porque las condiciones de operación incluyen un alto estrés mecánico y bajas velocidades de rotación. Debido a que estas condiciones de operación impactan negativamente en los resultados de los métodos y técnicas estándar aplicados en los sistemas actuales de monitoreo basados en el estado, el mantenimiento basado en el estado de los rodamientos pitch sigue siendo un desafío. Por tanto, esta tesis se centra en la investigación de métodos y técnicas novedosas que obtengan información fiable sobre el estado de los rodamientos pitch para el mantenimiento basado en la condición. Inicialmente, se realiza el reconocimiento del estado del arte para reconocer los métodos y señales utilizados. Este paso avala la decisión de analizar las señales de vibración y las emisiones acústicas a lo largo de esta tesis. Debido a las condiciones de funcionamiento particulares de los rodamientos pitch, esta investigación reconoce la necesidad de crear un conjunto de datos para replicar las condiciones de funcionamiento particulares del rodamiento pitch en una experiencia de laboratorio controlado. Como resultado, se genera un conjunto de datos basado en vibraciones y un segundo conjunto de datos basado en emisiones acústicas. El conjunto de datos de vibraciones permite la validación de un algoritmo novedoso para el diagnóstico de rodamientos de baja velocidad, el cual se basa en el concepto de la entropía según la definición de Shannon y Rényi. En comparación con los métodos clásicos que se encuentran en la literatura, el diagnóstico de rodamientos de baja velocidad basado en indicadores basados en la entropía puede extraer información más confiable. Además, la investigación de la combinación de varios indicadores para mejorar el diagnóstico revela que los indicadores basados en la entropía pueden extraer más información que los indicadores habituales utilizados en la academia. El conjunto de datos de las emisiones acústicas de los rodamientos de baja velocidad ayuda a contribuir al desarrollo de métodos de diagnóstico. En esta investigación, el análisis de la energía de las señales revela una dependencia de la intensidad y la presencia de daño. Además, se enfatiza una relación entre la forma de onda de la energía analizada y la existencia de daño.L'energia eòlica és una de les principals energies renovables considerades per reemplaçar els combustibles fòssils en la generació d'electricitat a nivell mundial. Per millorar i accelerar la seva implementació a gran escala, és vital reduir els costos associats amb el manteniment. Com els trencaments dels components obliguen a la turbina a aturar-se durant llargs períodes de reparació, la industria eòlica necessita canviar de l'antiquat manteniment preventiu o correctiu al manteniment basat en la condició (també anomenat manteniment predictiu). El manteniment basat en la condició dels rodaments de pas és especialment desafiant, perquè les condicions d’operació inclouen un alt estrès mecànic i baixes velocitats de rotació. A causa de que aquestes condicions d’operació impacten negativament en els resultats dels mètodes i tècniques estàndard aplicats en els sistemes actuals de monitorització basats en l'estat, el manteniment basat en l'estat dels rodaments de pas segueix sent un desafiament. Per tant, aquesta tesi se centra en la investigació de mètodes i tècniques noves que obtinguin informació fiable sobre l'estat dels rodaments de pas per al manteniment basat en la condició. Inicialment, es realitza el reconeixement de l'estat de l'art per reconèixer els mètodes i senyals utilitzats. Aquest pas avala la decisió d'analitzar els senyals de vibració i les emissions acústiques al llarg d'aquesta tesi. A causa de les condicions de funcionament particulars dels rodaments de pas, aquesta investigació reconeix la necessitat de crear un conjunt de dades per replicar les condicions de funcionament particulars del rodament de pas en un experiment de laboratori controlat. Com a resultat, es genera un conjunt de dades basat en vibracions i un segon conjunt de dades basat en emissions acústiques. El conjunt de dades de vibracions permet la validació d'un algoritme nou per al diagnòstic de rodaments de baixa velocitat, el qual es basa en el concepte de l'entropia segons la definició de Shannon i Renyi. En comparació amb els mètodes clàssics que es troben a la literatura, el diagnòstic de rodaments de baixa velocitat basat en indicadors basats en l'entropia pot extreure informació més fiable. A més, la investigació de la combinació de diversos indicadors per millorar el diagnòstic revela que els indicadors basats en l'entropia poden extreure més informació que els indicadors habituals utilitzats en la literatura. El conjunt de dades de les emissions acústiques dels rodaments de baixa velocitat ajuda a contribuir al desenvolupament de mètodes de diagnòstic. En aquesta investigació, l’anàlisi de l'energia de les senyals revela una dependència de la intensitat i la presència de dany. A més, s'emfatitza una relació entre la forma d'ona de l'energia analitzada i l’existència de dany.Energia eolikoa mundu mailan elektrizitatea sortu eta erregai fosilak ordezkatzeko energia berriztagarri nagusietako bat da. Eskala handiko ezarpena hobetu eta bizkortzeko, ezinbestekoa da mantentze-lanekin lotutako kostuak murriztea. Osagaien hausturek turbina konponketa-aldi luzeetan gelditzera behartzen dutenez, industria eolikoak mantentze-lan prebentibo edo zuzentzaile zaharkitutik egoeran oinarritutako mantentzelanetara aldatu behar du (mantentze-lan prediktiboa ere esaten zaio). Pitch errodamenduen egoeran oinarritutako mantentzea bereziki desa atzailea da, tentsio mekaniko handiak jasaten baitituzte eta errotazio-abiadura txikietan egoten baitira abian. Operaziobaldintza horiek eragin negatiboa dutenez egoeran oinarritutako egungo monitorizazio sistemetan erabiltzen diren metodo eta teknika estandarren emaitzetan, pitch errodamenduen egoeran oinarritutako mantentze-lanak erronka bat izaten jarraitzen du. Tesi hau egoeran oinarritutako mantenurako pitch errodamenduen egoerari buruzko informazio dagarria lortzen duten metodo eta teknika berritzaileen ikerketan oinarritzen da. Hasieran, teknologiaren egungo egoera aztertzen da, erabilitako metodoak eta seinaleak ezagutzeko. Urrats honek tesi honetan zehar bibrazio-seinaleak eta emisio akustikoak aztertzeko erabakia bermatzen du. Pitch errodamenduen funtzionamendu baldintza bereziak direla eta, ikerketa honek adierazten du beharrezkoa dela datu multzo bat sortzea pitch errodamenduaren funtzionamendu baldintza partikularrak erreplikatzeko laborategi kontrolatuko testuinguru batean. Ondorioz, bibrazioetan oinarritutako datu-multzo bat eta emisio akustikoetan oinarritutako bigarren datu-multzo bat sortzen dira. Bibrazioen datu-multzoak abiadura txikiko errodamenduen diagnostikorako algoritmo berritzaile bat baliozkotzea ahalbidetzen du, zeina entropiaren kontzeptuan oinarritzen baita Shannon eta R enyiren de nizioaren arabera. Literaturan dauden metodo klasikoekin alderatuta, entropian oinarritutako adierazleek abiadura txikiko errodamenduen diagnostikorako informazio dagarriagoa atera dezakete. Gainera, diagnostikoa hobetzeko hainbat adierazleren konbinazioaren ikerketak agerian uzten du entropian oinarritutako adierazleek akademian erabiltzen diren ohiko adierazleek baino informazio gehiago atera dezaketela. Abiadura txikiko errodamenduen emisio akustikoen datu multzoak diagnostiko metodoak garatzen laguntzen du. Ikerketa lan honetan, seinaleen energiaren azterketak intentsitatearekiko eta kaltearen presentziarekiko dependentzia adierazten du. Gainera, aztertutako energiaren uhin-formaren eta kaltearen arteko erlazioa nabarmentzen da.Postprint (published version