Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination\u27s complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go

Speech Processes for Brain-Computer Interfaces

Speech interfaces have become widely used and are integrated in many applications and devices. However, speech interfaces require the user to produce intelligible speech, which might be hindered by loud environments, concern to bother bystanders or the general in- ability to produce speech due to disabilities. Decoding a usera s imagined speech instead of actual speech would solve this problem. Such a Brain-Computer Interface (BCI) based on imagined speech would enable fast and natural communication without the need to actually speak out loud. These interfaces could provide a voice to otherwise mute people. This dissertation investigates BCIs based on speech processes using functional Near In- frared Spectroscopy (fNIRS) and Electrocorticography (ECoG), two brain activity imaging modalities on opposing ends of an invasiveness scale. Brain activity data have low signal- to-noise ratio and complex spatio-temporal and spectral coherence. To analyze these data, techniques from the areas of machine learning, neuroscience and Automatic Speech Recog- nition are combined in this dissertation to facilitate robust classification of detailed speech processes while simultaneously illustrating the underlying neural processes. fNIRS is an imaging modality based on cerebral blood flow. It only requires affordable hardware and can be set up within minutes in a day-to-day environment. Therefore, it is ideally suited for convenient user interfaces. However, the hemodynamic processes measured by fNIRS are slow in nature and the technology therefore offers poor temporal resolution. We investigate speech in fNIRS and demonstrate classification of speech processes for BCIs based on fNIRS. ECoG provides ideal signal properties by invasively measuring electrical potentials artifact- free directly on the brain surface. High spatial resolution and temporal resolution down to millisecond sampling provide localized information with accurate enough timing to capture the fast process underlying speech production. This dissertation presents the Brain-to- Text system, which harnesses automatic speech recognition technology to decode a textual representation of continuous speech from ECoG. This could allow to compose messages or to issue commands through a BCI. While the decoding of a textual representation is unparalleled for device control and typing, direct communication is even more natural if the full expressive power of speech - including emphasis and prosody - could be provided. For this purpose, a second system is presented, which directly synthesizes neural signals into audible speech, which could enable conversation with friends and family through a BCI. Up to now, both systems, the Brain-to-Text and synthesis system are operating on audibly produced speech. To bridge the gap to the final frontier of neural prostheses based on imagined speech processes, we investigate the differences between audibly produced and imagined speech and present first results towards BCI from imagined speech processes. This dissertation demonstrates the usage of speech processes as a paradigm for BCI for the first time. Speech processes offer a fast and natural interaction paradigm which will help patients and healthy users alike to communicate with computers and with friends and family efficiently through BCIs

Convolutional Neural Network for Functional Near-Infrared Spectroscopy-Based Brain-Computer Interface

Brain-computer interface (BCI) is a communication system that translates the brain signal directly to a computer or external devices. It is a promising solution for the patients with neurological disorders as the system is able to restore the movement ability. Various neuroimaging modalities have been utilized for brain signal acquisition, however, functional near-infrared spectroscopy (fNIRS) provides many advantages over other modalities. Hence, it has gained attention for implementing in BCI system. For developing BCI system, the appropriate machine learning algorithm and discriminating features from the hemodynamic response signal are desired, as the previous studies have reported the performance enhancement of fNIRS-based BCI in terms of classification accuracy by focusing on the classifier as well as signal features. The aim of this thesis is to improve the classification accuracy in fNIRS-based BCI by classifying and extracting feature automatically. The convolutional neural network (CNN) was applied owing to the automatic feature extractor and classifier instead of manual feature extraction in the conventional methods. In the experiment, four healthy subjects were measured the hemodynamic response signal evoked by performing tasks including rest, right and left hand motor executions. The conventional methods of fNIRS-based BCI using signal mean, slope, peak, variance, skewness, and kurtosis as the features, and support vector machine (SVM) and artificial neural network (ANN) as the classification methods were compared with CNN-based method. The results show the improvement of classification accuracy of CNN-based method over SVM-based and ANN-based method 6.92% and 3.75%, respectively. The main contributions of this thesis are (1) the promising feature extraction and classification method for fNIRS-based BCI using CNN and (2) the analysis of the feature extracted by conventional methods and convolutional filter of the CNN. ⓒ 2017 DGISTprohibitionI. INTRODUCTION 1-- 1. Motivation 1-- 2. Objective 2-- II. BACKGROUND AND RELATEDWORK 4-- 1. Functional Near-Infrared Spectroscopy (fNIRS) 4-- 2. fNIRS-based BCI 5-- 3. Feature Extraction and Classification 6-- 3.1 Feature Extraction 6-- 3.2 Support Vector Machine (SVM) 7-- 3.3 Artificial Neural Network (ANN) 7-- 3.4 Convolutional Neural Network (CNN) 9-- 4. Evaluation 11-- III. METHOD 12-- 1. Participants 12-- 2. Data Acquisition 12-- 3. Experimental Procedure 12-- 4. Preprocessing 13-- 4.1 Concentration Changes of Hemoglobin 13-- 4.2 Filtering 14-- 5. Feature Extraction and Classification 16-- 5.1 Conventional Method 16-- 5.2 Proposed Structures of CNN 17-- 6. Feature Visualization 20-- IV. RESULTS AND DISCUSSIONS 23-- 1. Measured Hemodynamic Responses 23-- 2. Classification Accuracy 24-- 3. Feature Visualization 26-- 4. Future Work 28-- V. CONCLUSION 30-- References 31-- Acknowledgments 38-- Curriculum Vitae 39MasterdCollectio

Association between Prefrontal fNIRS signals during Cognitive tasks and College scholastic ability test (CSAT) scores: Analysis using a quantum annealing approach

Academic achievement is a critical measure of intellectual ability, prompting extensive research into cognitive tasks as potential predictors. Neuroimaging technologies, such as functional near-infrared spectroscopy (fNIRS), offer insights into brain hemodynamics, allowing understanding of the link between cognitive performance and academic achievement. Herein, we explored the association between cognitive tasks and academic achievement by analyzing prefrontal fNIRS signals. A novel quantum annealer (QA) feature selection algorithm was applied to fNIRS data to identify cognitive tasks correlated with CSAT scores. Twelve features (signal mean, median, variance, peak, number of peaks, sum of peaks, slope, minimum, kurtosis, skewness, standard deviation, and root mean square) were extracted from fNIRS signals at two time windows (10- and 60-second) to compare results from various feature variable conditions. The feature selection results from the QA-based and XGBoost regressor algorithms were compared to validate the former's performance. In a three-step validation process using multiple linear regression models, correlation coefficients between the feature variables and the CSAT scores, model fitness (adjusted R2), and model prediction error (RMSE) values were calculated. The quantum annealer demonstrated comparable performance to classical machine learning models, and specific cognitive tasks, including verbal fluency, recognition, and the Corsi block tapping task, were correlated with academic achievement. Group analyses revealed stronger associations between Tower of London and N-back tasks with higher CSAT scores. Quantum annealing algorithms have significant potential in feature selection using fNIRS data, and represents a novel research approach. Future studies should explore predictors of academic achievement and cognitive ability.Comment: 42 pages, 11 table