Transfer learning for a multimodal hybrid EEG-FTCD Brain-Computer Interface

Transfer learning has been used to overcome the limitations of machine learning in Brain-Computer Interface (BCI) applications. Transfer learning aims to provide higher performance than no-transfer machine learning when only a limited number of training data is available and can consequently reduce training and calibration requirements. BCI systems are designed to provide communication and control tools for individuals with limited speech and physical abilities (LSPA). Most noninvasive BCI systems are based on Electroencephalogram (EEG) because of EEG \textquotesingle s cost effectiveness and portability. However, EEG signals present low signal-to-noise ratio and nonstationarity due to background brain activity. Such a behavior may decrease the global performance of the system. To overcome the disadvantages of EEG signals, in our previous work, we developed two different multi-modal BCI systems based on EEG and functional transcranial Doppler (fTCD), a cerebral flood velocity measure. These two multi-modal systems that combine EEG and fTCD signals aim to reduce performance degradation obtained when EEG was the only BCI modality. One of the systems is based on steady state evoked potentials and the other one is designed using motor imagery paradigms. Our results have shown that such a hybrid system outperforms EEG only BCIs. However, both systems require significant amount of training data for personalized design which could be tiresome for the target population. In this study, we extend these systems by performing a new transfer learning algorithm and we demonstrate the corresponding algorithm on the three different binary classification tasks for both BCIs in order to reduce the calibration requirements. Performing experiments with healthy participants, we collected EEG and fTCD data using both BCI systems. In order to apply transfer learning and to reduce the calibration requirements for BCIs, for each participant, we identify the most informative datasets from the rest of the participants based on probabilistic similarities between the class conditional distributions and increase the training set from this data. We demonstrate that transfer learning reduces the calibration requirements up to \%87.5 for BCI systems. Also, through comparison between different classifiers LDA, QDA, and SVM, we observe that QDA achieves the higher difference between transfer learning and no transfer accuracy

A Hybrid Brain-Computer Interface Based on Electroencephalography and Functional Transcranial Doppler Ultrasound

Hybrid brain computer interfaces (BCIs) combining multiple brain imaging modalities have been proposed recently to boost the performance of single modality BCIs. We advance the state of hybrid BCIs by introducing a novel system that measures electrical brain activity as well as cerebral blood flow velocity using Electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD), respectively. The system we developed employs two different paradigms to induce changes simultaneously in EEG and fTCD and to infer user intent. One of these paradigms includes visual stimuli to simultaneously induce steady state visually evoked potentials (SSVEPs) and instructs users to perform word generation (WG) and mental rotation (MR) tasks, while the other paradigm instructs users to perform left and right arm motor imagery (MI) tasks through visual stimuli. To improve accuracy and information transfer rate (ITR) of the proposed system compared to those obtained through our preliminary analysis, using classical feature extraction approaches, we mainly contribute to multi-modal fusion of EEG and fTCD features. Specifically, we proposed a probabilistic fusion of EEG and fTCD evidences instead of simple concatenation of EEG and fTCD feature vectors that we performed in our preliminary analysis. Experimental results showed that the MI paradigm outperformed the MR/WG one in terms of both accuracy and ITR. In particular, 93.85%, 93.71%, and 100% average accuracies and 19.89, 26.55, and 40.83 bits/min v average ITRs were achieved for right MI vs baseline, left MI versus baseline, and right MI versus left MI, respectively. Moreover, for both paradigms, the EEG-fTCD BCI with the proposed analysis techniques outperformed all EEG- fNIRS BCIs in terms of accuracy and ITR. In addition, to investigate the feasibility of increasing the possible number of BCI commands, we extended our approaches to solve the 3-class problems for both paradigms. It was found that the MI paradigm outperformed the MR/WG paradigm and achieved 96.58% average accuracy and 45 bits/min average ITR. Finally, we introduced a transfer learning approach to reduce the calibration requirements of the proposed BCI. This approach was found to be very efficient especially with the MI paradigm as it reduced the calibration requirements by at least 60.43%