Study of non-invasive cognitive tasks and feature extraction techniques for brain-computer interface (BCI) applications

A brain-computer interface (BCI) provides an important alternative for disabled people that enables the non-muscular communication pathway among individual thoughts and different assistive appliances. A BCI technology essentially consists of data acquisition, pre-processing, feature extraction, classification and device command. Indeed, despite the valuable and promising achievements already obtained in every component of BCI, the BCI field is still a relatively young research field and there is still much to do in order to make BCI become a mature technology. To mitigate the impediments concerning BCI, the study of cognitive task together with the EEG feature and classification framework have been investigated. There are four distinct experiments have been conducted to determine the optimum solution to those specific issues. In the first experiment, three cognitive tasks namely quick math solving, relaxed and playing games have been investigated. The features have been extracted using power spectral density (PSD), logenergy entropy, and spectral centroid and the extracted feature has been classified through the support vector machine (SVM), K-nearest neighbor (K-NN), and linear discriminant analysis (LDA). In this experiment, the best classification accuracy for single channel and five channel datasets were 86% and 91.66% respectively that have been obtained by the PSD-SVM approach. The wink based facial expressions namely left wink, right wink and no wink have been studied through fast Fourier transform (FFT) and sample range feature and then the extracted features have been classified using SVM, K-NN, and LDA. The best accuracy (98.6%) has been achieved by the sample range-SVM based approach. The eye blinking based facial expression has been investigated following the same methodology as the study of wink based facial expression. Moreover, the peak detection approach has also been employed to compute the number of blinks. The optimum accuracy of 99% has been achieved using the peak detection approach. Additionally, twoclass motor imagery hand movement has been classified using SVM, K-NN, and LDA where the feature has been extracted through PSD, spectral centroid and continuous wavelet transform (CWT). The optimum 74.7% accuracy has been achieved by the PSDSVM approach. Finally, two device command prototypes have been designed to translate the classifier output. One prototype can translate four types of cognitive tasks in terms of 5 watts four different colored bulbs, whereas, another prototype may able to control DC motor utilizing cognitive tasks. This study has delineated the implementation of every BCI component to facilitate the application of brainwave assisted assistive appliances. Finally, this thesis comes to the end by drawing the future direction regarding the current issues of BCI technology and these directions may significantly enhance usability for the implementation of commercial applications not only for the disabled but also for a significant number of healthy users

Investigating the possibilities of using singing imagery to enhance EEG-based active BCIs

Active brain-computer interfaces are a novel communication/interaction pathway that relies on the performance of imagery tasks. The patterns of brain activity associated with these tasks are detected and decoded as commands for operating an external device. This study focuses on augmenting the practicality of such systems by investigating the mental task of singing imagery. Singing imagery is the simple act of imagining singing a song in your head. Despite its straightforward nature, the potential of singing imagery as an alternative task for active BCIs or for increasing their number of commands has yet to be thoroughly investigated. The research described in this thesis comprises two phases. In the first study, singing imagery is combined with the commonly used imagery tasks in BCI research (i.e., 4- and 5-class combinations consisting of the imagined movement of the left hand, right hand, feet, and tongue, as well as a \rest" state). Filter bank common spatial patterns algorithm and the random forest classifier are utilized to incorporate a singing imagery task in the 2-, 3-, 4-, and 5-class combinations. These analyses resulted in comparable classification accuracies to conventional motor imagery tasks. Hence, based on the survey results, singing imagery could be considered as a potentially more intuitive alternative mental task. Furthermore, singing imagery may also be a practical approach for increasing the number of commands to six, where accuracies as high as 60.7% were achieved. The second study investigated the potential of using \dual imagery" tasks (i.e., the simultaneous performance of two single tasks, in this case, singing imagery and one of the conventional motor imagery tasks) as additional BCI control tasks. Here, the 3- and 4-class analyses of the dual tasks and their constituent single tasks (alongside a \rest" state for the 4-class) were carried out to verify the possibility of differentiating them. Using an extended version of filter bank common spatial patterns and regularized linear discriminant analysis classifiers, average accuracies as high as 64.1% and 63% were achieved for the 3, and 4-class scenarios, respectively. Next, the dual imagery tasks were combined with conventional single motor imagery tasks to investigate increasing the number of commands to seven or eight. As a result, for the 7- and 8-class scenarios, accuracies as high as 55.4%, and 50.5%, which are well above the corresponding chance levels of 14.3% and 12.5%, were obtained. Increasing the number of commands a BCI can recognize is important as it can significantly impact the user's experience with the device. Specifically, a BCI with a more intuitive list of commands can help the user avoid a high mental workload. Moreover, a higher number of commands can be helpful by allowing users to communicate with a higher information transfer rate. Based on the results of this thesis research, singing imagery appears to be a potentially viable solution for improving active BCIs

Intelligent Biosignal Processing in Wearable and Implantable Sensors

This reprint provides a collection of papers illustrating the state-of-the-art of smart processing of data coming from wearable, implantable or portable sensors. Each paper presents the design, databases used, methodological background, obtained results, and their interpretation for biomedical applications. Revealing examples are brain–machine interfaces for medical rehabilitation, the evaluation of sympathetic nerve activity, a novel automated diagnostic tool based on ECG data to diagnose COVID-19, machine learning-based hypertension risk assessment by means of photoplethysmography and electrocardiography signals, Parkinsonian gait assessment using machine learning tools, thorough analysis of compressive sensing of ECG signals, development of a nanotechnology application for decoding vagus-nerve activity, detection of liver dysfunction using a wearable electronic nose system, prosthetic hand control using surface electromyography, epileptic seizure detection using a CNN, and premature ventricular contraction detection using deep metric learning. Thus, this reprint presents significant clinical applications as well as valuable new research issues, providing current illustrations of this new field of research by addressing the promises, challenges, and hurdles associated with the synergy of biosignal processing and AI through 16 different pertinent studies. Covering a wide range of research and application areas, this book is an excellent resource for researchers, physicians, academics, and PhD or master students working on (bio)signal and image processing, AI, biomaterials, biomechanics, and biotechnology with applications in medicine

A novel EEG based linguistic BCI

While a human being can think coherently, physical limitations no matter how severe, should never become disabling. Thinking and cognition are performed and expressed through language, which is the most natural form of human communication. The use of covert speech tasks for BCIs has been successfully achieved for invasive and non-invasive systems. In this work, by incorporating the most recent discoveries on the spatial, temporal, and spectral signatures of word production, a novel system is designed, which is custom-build for linguistic tasks. Other than paying attention and waiting for the onset cue, this BCI requires absolutely no cognitive effort from the user and operates using automatic linguistic functions of the brain in the first 312ms post onset, which is also completely out of the control of the user and immune from inconsistencies. With four classes, this online BCI achieves classification accuracy of 82.5%. Each word produces a signature as unique as its phonetic structure, and the number of covert speech tasks used in this work is limited by computational power. We demonstrated that this BCI can successfully use wireless dry electrode EEG systems, which are becoming as capable as traditional laboratory grade systems. This frees the potential user from the confounds of the lab, facilitating real-world application. Considering that the number of words used in daily life does not exceed 2000, the number of words used by this type of novel BCI may indeed reach this number in the future, with no need to change the current system design or experimental protocol. As a promising step towards noninvasive synthetic telepathy, this system has the potential to not only help those in desperate need, but to completely change the way we communicate with our computers in the future as covert speech is much easier than any form of manual communication and control

Binary visual imagery discriminator from EEG signals based on convolutional neural networks

[EN] A Brain-Computer Intarface (BCI) is a technology that allows direct communication between the brain and the outside world without the need to use the peripheral nervous system. Most BCI systems focus on the use of motor imagination, evoked potentials, or slow cortical rhythms. In this work, the possibility of using visual imagination to construct a binary discriminator has been studied. EEG signals from seven people have been recorded while imagining seven geometric figures. Using convolutional neural networks it has been possible to distinguish between the imagination of a geometric figure and relaxation with an average success rate of 91 % with a Cohen kappa value of 0.77 and a percentage of false positives of 9 %.[ES] Las interfaces cerebro-máquina (Brain-Computer Intarface, BCI, en inglés) son una tecnología que permite la comunicación directa entre el cerebro y el mundo exterior sin necesidad de utilizar el sistema nervioso periferico. La mayoría de sistemas BCI se centran en la utilización de la imaginación motora, los potenciales evocados o los ritmos corticales lentos. En este trabajo se ha estudiado la posibilidad de utilizar la imaginación visual para construir un discriminador binario (brain-switch, en inglés). Concretamente, a partir del registro de señales EEG de siete personas mientras imaginaban siete figuras geométricas, se ha desarrollado un BCI basado en redes neuronales convolucionales y en la densidad de potencia espectral en la banda α (8-12 Hz), que ha conseguido distinguir entre la imaginación de una figura geométrica cualquiera y el relax, con un acierto promedio del 91 %, con un valor Kappa de Cohen de 0.77 y un porcentaje de falsos positivos del 9 %.Este trabajo ha sido parcialmente financiado por el proyecto TIN2017-88515-C2-2-R del Ministerio de Economía y Competitividad.Llorella, FR.; Iáñez, E.; Azorín, JM.; Patow, G. (2021). Discriminador binario de imaginación visual a partir de señales EEG basado en redes neuronales convolucionales. Revista Iberoamericana de Automática e Informática industrial. 19(1):108-116. https://doi.org/10.4995/riai.2021.14987OJS108116191Abhiram, Singh, A., Gumaste, 2019. Decoding imagined speech and computer control using brain waves. arXiv.Aggarwal, S., Chugh, N., Jan. 2019. Signal processing techniques for motor imagery brain computer interface: A review. Array 1-2, 100003. https://doi.org/10.1016/j.array.2019.100003Ahn, M., Lee, M., Choi, J., Jun, S., Aug. 2014. A review of brain-computer interface games and an opinion survey from researchers, developers and users. Sensors 14 (8), 14601-14633. https://doi.org/10.3390/s140814601Altman, D. G., 2006. Practical Statistics for Medical Research. Chapman & Hall/CRC.Anderson, C. W., Sijercic, Z., 1996. Classification of eeg signals from four subjects during five mental tasks. In: Solving engineering problems with neural networks: proceedings of the conference on engineering applications in neural networks (EANN'96). Turkey, pp. 407-414.Bigdely-Shamlo, N., Mullen, T., Kothe, C., Su, K.-M., Robbins, K. A., Jun.2015. The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Frontiers in Neuroinformatics 9. https://doi.org/10.3389/fninf.2015.00016Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E., Flor, H., Mar. 1999. A spelling device for the paralysed. Nature 398 (6725), 297-298. https://doi.org/10.1038/18581Bobrov, P., Frolov, A., Cantor, C., Fedulova, I., Bakhnyan, M., Zhavoronkov, A., Jun. 2011. Brain-computer interface based on generation of visual images. PLoS ONE 6 (6), e20674. https://doi.org/10.1371/journal.pone.0020674Craik, A., He, Y., Contreras-Vidal, J. L., Apr. 2019. Deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of Neural Engineering 16 (3), 031001. https://doi.org/10.1088/1741-2552/ab0ab5Esfahani, E. T., Sundararajan, V., Oct. 2012. Classification of primitive shapes using brain-computer interfaces. Computer-Aided Design 44 (10), 1011-1019. https://doi.org/10.1016/j.cad.2011.04.008Fernando, L., Nicolas-Alonso, J., Gomez-Gil, 2012. Brain computer interface, a review. Sensors 12 (2), 1211-1279. https://doi.org/10.3390/s120201211Gavali, P., Banu, J. S., 2019. Deep convolutional neural network for image classification on CUDA platform. In: Deep Learning and Parallel Computing Environment for Bioengineering Systems. Elsevier, pp. 99-122. https://doi.org/10.1016/B978-0-12-816718-2.00013-0Gong, M., Xu, G., Li, M., Lin, F., May 2020. An idle state-detecting method based on transient visual evoked potentials for an asynchronous ERP-based BCI. Journal of Neuroscience Methods 337, 108670. https://doi.org/10.1016/j.jneumeth.2020.108670Han, C.-H., Muller, K.-R., Hwang, H.-J., Mar. 2020. Brain-switches for asynchronous brain-computer interfaces: A systematic review. Electronics 9 (3), 422. https://doi.org/10.3390/electronics9030422Hortal, E., Planelles, D., Resquin, F., Climent, J. M., Azorín, J. M., Pons, J. L., Oct. 2015. Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions. Journal of NeuroEngineering and Rehabilitation 12 (1). https://doi.org/10.1186/s12984-015-0082-9Jiang, J., Zhou, Z., Yin, E., Yu, Y., Liu, Y., Hu, D., Nov. 2015. A novel morse code-inspired method for multiclass motor imagery brain-computer interface (BCI) design. Computers in Biology and Medicine 66, 11-19. https://doi.org/10.1016/j.compbiomed.2015.08.011Jurcak, V., Tsuzuki, D., Dan, I., Feb. 2007. 10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems. NeuroImage 34 (4), 1600-1611. https://doi.org/10.1016/j.neuroimage.2006.09.024Kim, C., Sun, J., Liu, D., Wang, Q., Paek, S., Mar. 2018. An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI. Medical & Biological Engineering & Computing 56 (9), 1645-1658. https://doi.org/10.1007/s11517-017-1761-4Knauff, M., Kassubek, J., Mulack, T., Greenlee, M. W., Dec. 2000. Cortical activation evoked by visual mental imagery as measured by fMRI. NeuroReport 11 (18), 3957-3962. https://doi.org/10.1097/00001756-200012180-00011Kosmyna, N., Lindgren, J. T., Lecuyer, A., Sep. 2018. Attending to visual stimuli versus performing visual imagery as a control strategy for EEG-based brain-computer interfaces. Scientific Reports 8 (1). https://doi.org/10.1038/s41598-018-31472-9Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., Arnaldi, B., Jan. 2007. A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering 4 (2), R1-R13. https://doi.org/10.1088/1741-2560/4/2/R01McCall, J., Dec. 2005. Genetic algorithms for modelling and optimisation. Journal of Computational and Applied Mathematics 184 (1), 205-222. https://doi.org/10.1016/j.cam.2004.07.034Melnik, A., Legkov, P., Izdebski, K., Karcher, S. M., Hairston, W. D., Ferris, D. P., Konig, P., Mar. 2017. Systems, subjects, sessions: To what extent do these factors influence EEG data? Frontiers in Human Neuroscience 11. https://doi.org/10.3389/fnhum.2017.00150Pedroni, A., Bahreini, A., Langer, N., Nov. 2019. Automagic: Standardized preprocessing of big EEG data. bioRxiv. https://doi.org/10.1101/460469Planelles, D., Hortal, E., Costa, Á., Úbeda, A., Iáez, E., Azorín, J., Sep. 2014. Evaluating classifiers to detect arm movement intention from EEG signals. Sensors 14 (10), 18172-18186. https://doi.org/10.3390/s141018172Proakis, John, M., Dimitri, 1996. Digital signal processing: principles, algorithms and applications. Prentice-Hall, 910-913.Reza, Abiri, S., Borhani, E. W., Sellers, Y., Jiang, X., Zhao, 2018. A comprehensive review of eeg-based brain-computer interface paradigms. Journal of Neural Engineering 16. https://doi.org/10.1088/1741-2552/aaf12eSeeck, M., Koessler, L., Bast, T., Leijten, F., Michel, C., Baumgartner, C., He, B., Beniczky, S., Oct. 2017. The standardized EEG electrode array of the IFCN. Clinical Neurophysiology 128 (10), 2070-2077. https://doi.org/10.1016/j.clinph.2017.06.254Swets, J., 2014. Signal detection theory and ROC analysis in psychology and diagnostics : collected papers. Psychology Press, New York, New York Hove, England. https://doi.org/10.4324/9781315806167Xie, S., Kaiser, D., Cichy, R. M., Aug. 2020. Visual imagery and perception share neural representations in the alpha frequency band. Current Biology 30 (15), 3062. https://doi.org/10.1016/j.cub.2020.07.023Zhang, W., Tan, C., Sun, F., Wu, H., Zhang, B., Dec. 2018. A review of EEGbased brain-computer interface systems design. Brain Science Advances 4 (2), 156-167. https://doi.org/10.26599/BSA.2018.905001

Development of A Versatile Multichannel CWNIRS Instrument for Optical Brain-Computer Interface Applications

This thesis describes the design, development, and implementation of a versatile multichannel continuous-wave near-infrared spectroscopy (CWNIRS) instrument for brain-computer interface (BCI) applications. Specifically, it was of interest to assess what gains could be achieved by using a multichannel device compared to the single channel device implemented by Coyle in 2004. Moreover, the multichannel approach allows for the assessment of localisation of functional tasks in the cerebral cortex, and can identify lateralisation of haemodynamic responses to motor events. The approach taken to extend single channel to multichannel was based on a software-controlled interface. This interface allowed flexibility in the control of individual optodes including their synchronisation and modulation (AM, TDM, CDMA). Furthermore, an LED driver was developed for custom-made triple-wavelength LEDs. The system was commissioned using a series of experiments to verify the performance of individual components in the system. The system was then used to carry out a set of functional studies including motor imagery and cognitive tasks. The experimental protocols based on motor imagery and overt motor tasks were verified by comparison with fMRI. The multichannel approach identified stroke rehabilitation as a new application area for optical BCI. In addition, concentration changes in deoxyhaemoglobin were identified as being a more localised indicator of functional activity, which is important for effective BCI design. An assessment was made on the effect of the duration of the stimulus period on the haemodynamic signals. This demonstrated the possible benefits of using a shorter stimulus period to reduce the adverse affects of low blood pressure oscillations. i

Development of A Versatile Multichannel CWNIRS Instrument for Optical Brain-Computer Interface Applications

This thesis describes the design, development, and implementation of a versatile multichannel continuous-wave near-infrared spectroscopy (CWNIRS) instrument for brain-computer interface (BCI) applications. Specifically, it was of interest to assess what gains could be achieved by using a multichannel device compared to the single channel device implemented by Coyle in 2004. Moreover, the multichannel approach allows for the assessment of localisation of functional tasks in the cerebral cortex, and can identify lateralisation of haemodynamic responses to motor events. The approach taken to extend single channel to multichannel was based on a software-controlled interface. This interface allowed flexibility in the control of individual optodes including their synchronisation and modulation (AM, TDM, CDMA). Furthermore, an LED driver was developed for custom-made triple-wavelength LEDs. The system was commissioned using a series of experiments to verify the performance of individual components in the system. The system was then used to carry out a set of functional studies including motor imagery and cognitive tasks. The experimental protocols based on motor imagery and overt motor tasks were verified by comparison with fMRI. The multichannel approach identified stroke rehabilitation as a new application area for optical BCI. In addition, concentration changes in deoxyhaemoglobin were identified as being a more localised indicator of functional activity, which is important for effective BCI design. An assessment was made on the effect of the duration of the stimulus period on the haemodynamic signals. This demonstrated the possible benefits of using a shorter stimulus period to reduce the adverse affects of low blood pressure oscillations. i

Smart Sensors for Healthcare and Medical Applications

This book focuses on new sensing technologies, measurement techniques, and their applications in medicine and healthcare. Specifically, the book briefly describes the potential of smart sensors in the aforementioned applications, collecting 24 articles selected and published in the Special Issue “Smart Sensors for Healthcare and Medical Applications”. We proposed this topic, being aware of the pivotal role that smart sensors can play in the improvement of healthcare services in both acute and chronic conditions as well as in prevention for a healthy life and active aging. The articles selected in this book cover a variety of topics related to the design, validation, and application of smart sensors to healthcare

Addressing the challenges posed by human machine interfaces based on force sensitive resistors for powered prostheses

Despite the advancements in the mechatronics aspect of prosthetic devices, prostheses control still lacks an interface that satisfies the needs of the majority of users. The research community has put great effort into the advancements of prostheses control techniques to address users’ needs. However, most of these efforts are focused on the development and assessment of technologies in the controlled environments of laboratories. Such findings do not fully transfer to the daily application of prosthetic systems. The objectives of this thesis focus on factors that affect the use of Force Myography (FMG) controlled prostheses in practical scenarios. The first objective of this thesis assessed the use of FMG as an alternative or synergist Human Machine Interface (HMI) to the more traditional HMI, i.e. surface Electromyography (sEMG). The assessment for this study was conducted in conditions that are relatively close to the real use case of prosthetic applications. The HMI was embedded in the custom prosthetic prototype that was developed for the pilot participant of the study using an off-the-shelf prosthetic end effector. Moreover, prostheses control was assessed as the user moved their limb in a dynamic protocol.The results of the aforementioned study motivated the second objective of this thesis: to investigate the possibility of reducing the complexity of high density FMG systems without sacrificing classification accuracies. This was achieved through a design method that uses a high density FMG apparatus and feature selection to determine the number and location of sensors that can be eliminated without significantly sacrificing the system’s performance. The third objective of this thesis investigated two of the factors that contribute to increased errors in force sensitive resistor (FSR) signals used in FMG controlled prostheses: bending of force sensors and variations in the volume of the residual limb. Two studies were conducted that proposed solutions to mitigate the negative impact of these factors. The incorporation of these solutions into prosthetic devices is discussed in these studies.It was demonstrated that FMG is a promising HMI for prostheses control. The facilitation of pattern recognition with FMG showed potential for intuitive prosthetic control. Moreover, a method for the design of a system that can determine the required number of sensors and their locations on each individual to achieve a simpler system with comparable performance to high density FMG systems was proposed and tested. The effects of the two factors considered in the third objective were determined. It was also demonstrated that the proposed solutions in the studies conducted for this objective can be used to increase the accuracy of signals that are commonly used in FMG controlled prostheses

Shortest Route at Dynamic Location with Node Combination-Dijkstra Algorithm

Abstract— Online transportation has become a basic requirement of the general public in support of all activities to go to work, school or vacation to the sights. Public transportation services compete to provide the best service so that consumers feel comfortable using the services offered, so that all activities are noticed, one of them is the search for the shortest route in picking the buyer or delivering to the destination. Node Combination method can minimize memory usage and this methode is more optimal when compared to A* and Ant Colony in the shortest route search like Dijkstra algorithm, but can’t store the history node that has been passed. Therefore, using node combination algorithm is very good in searching the shortest distance is not the shortest route. This paper is structured to modify the node combination algorithm to solve the problem of finding the shortest route at the dynamic location obtained from the transport fleet by displaying the nodes that have the shortest distance and will be implemented in the geographic information system in the form of map to facilitate the use of the system. Keywords— Shortest Path, Algorithm Dijkstra, Node Combination, Dynamic Location (key words