Binary visual imagery discriminator from EEG signals based on convolutional neural networks

[EN] A Brain-Computer Intarface (BCI) is a technology that allows direct communication between the brain and the outside world without the need to use the peripheral nervous system. Most BCI systems focus on the use of motor imagination, evoked potentials, or slow cortical rhythms. In this work, the possibility of using visual imagination to construct a binary discriminator has been studied. EEG signals from seven people have been recorded while imagining seven geometric figures. Using convolutional neural networks it has been possible to distinguish between the imagination of a geometric figure and relaxation with an average success rate of 91 % with a Cohen kappa value of 0.77 and a percentage of false positives of 9 %.[ES] Las interfaces cerebro-máquina (Brain-Computer Intarface, BCI, en inglés) son una tecnología que permite la comunicación directa entre el cerebro y el mundo exterior sin necesidad de utilizar el sistema nervioso periferico. La mayoría de sistemas BCI se centran en la utilización de la imaginación motora, los potenciales evocados o los ritmos corticales lentos. En este trabajo se ha estudiado la posibilidad de utilizar la imaginación visual para construir un discriminador binario (brain-switch, en inglés). Concretamente, a partir del registro de señales EEG de siete personas mientras imaginaban siete figuras geométricas, se ha desarrollado un BCI basado en redes neuronales convolucionales y en la densidad de potencia espectral en la banda α (8-12 Hz), que ha conseguido distinguir entre la imaginación de una figura geométrica cualquiera y el relax, con un acierto promedio del 91 %, con un valor Kappa de Cohen de 0.77 y un porcentaje de falsos positivos del 9 %.Este trabajo ha sido parcialmente financiado por el proyecto TIN2017-88515-C2-2-R del Ministerio de Economía y Competitividad.Llorella, FR.; Iáñez, E.; Azorín, JM.; Patow, G. (2021). Discriminador binario de imaginación visual a partir de señales EEG basado en redes neuronales convolucionales. Revista Iberoamericana de Automática e Informática industrial. 19(1):108-116. https://doi.org/10.4995/riai.2021.14987OJS108116191Abhiram, Singh, A., Gumaste, 2019. Decoding imagined speech and computer control using brain waves. arXiv.Aggarwal, S., Chugh, N., Jan. 2019. Signal processing techniques for motor imagery brain computer interface: A review. Array 1-2, 100003. https://doi.org/10.1016/j.array.2019.100003Ahn, M., Lee, M., Choi, J., Jun, S., Aug. 2014. A review of brain-computer interface games and an opinion survey from researchers, developers and users. Sensors 14 (8), 14601-14633. https://doi.org/10.3390/s140814601Altman, D. G., 2006. Practical Statistics for Medical Research. Chapman & Hall/CRC.Anderson, C. W., Sijercic, Z., 1996. Classification of eeg signals from four subjects during five mental tasks. In: Solving engineering problems with neural networks: proceedings of the conference on engineering applications in neural networks (EANN'96). Turkey, pp. 407-414.Bigdely-Shamlo, N., Mullen, T., Kothe, C., Su, K.-M., Robbins, K. A., Jun.2015. The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Frontiers in Neuroinformatics 9. https://doi.org/10.3389/fninf.2015.00016Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E., Flor, H., Mar. 1999. A spelling device for the paralysed. Nature 398 (6725), 297-298. https://doi.org/10.1038/18581Bobrov, P., Frolov, A., Cantor, C., Fedulova, I., Bakhnyan, M., Zhavoronkov, A., Jun. 2011. Brain-computer interface based on generation of visual images. PLoS ONE 6 (6), e20674. https://doi.org/10.1371/journal.pone.0020674Craik, A., He, Y., Contreras-Vidal, J. L., Apr. 2019. Deep learning for electroencephalogram (EEG) classification tasks: a review. Journal of Neural Engineering 16 (3), 031001. https://doi.org/10.1088/1741-2552/ab0ab5Esfahani, E. T., Sundararajan, V., Oct. 2012. Classification of primitive shapes using brain-computer interfaces. Computer-Aided Design 44 (10), 1011-1019. https://doi.org/10.1016/j.cad.2011.04.008Fernando, L., Nicolas-Alonso, J., Gomez-Gil, 2012. Brain computer interface, a review. Sensors 12 (2), 1211-1279. https://doi.org/10.3390/s120201211Gavali, P., Banu, J. S., 2019. Deep convolutional neural network for image classification on CUDA platform. In: Deep Learning and Parallel Computing Environment for Bioengineering Systems. Elsevier, pp. 99-122. https://doi.org/10.1016/B978-0-12-816718-2.00013-0Gong, M., Xu, G., Li, M., Lin, F., May 2020. An idle state-detecting method based on transient visual evoked potentials for an asynchronous ERP-based BCI. Journal of Neuroscience Methods 337, 108670. https://doi.org/10.1016/j.jneumeth.2020.108670Han, C.-H., Muller, K.-R., Hwang, H.-J., Mar. 2020. Brain-switches for asynchronous brain-computer interfaces: A systematic review. Electronics 9 (3), 422. https://doi.org/10.3390/electronics9030422Hortal, E., Planelles, D., Resquin, F., Climent, J. M., Azorín, J. M., Pons, J. L., Oct. 2015. Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions. Journal of NeuroEngineering and Rehabilitation 12 (1). https://doi.org/10.1186/s12984-015-0082-9Jiang, J., Zhou, Z., Yin, E., Yu, Y., Liu, Y., Hu, D., Nov. 2015. A novel morse code-inspired method for multiclass motor imagery brain-computer interface (BCI) design. Computers in Biology and Medicine 66, 11-19. https://doi.org/10.1016/j.compbiomed.2015.08.011Jurcak, V., Tsuzuki, D., Dan, I., Feb. 2007. 10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems. NeuroImage 34 (4), 1600-1611. https://doi.org/10.1016/j.neuroimage.2006.09.024Kim, C., Sun, J., Liu, D., Wang, Q., Paek, S., Mar. 2018. An effective feature extraction method by power spectral density of EEG signal for 2-class motor imagery-based BCI. Medical & Biological Engineering & Computing 56 (9), 1645-1658. https://doi.org/10.1007/s11517-017-1761-4Knauff, M., Kassubek, J., Mulack, T., Greenlee, M. W., Dec. 2000. Cortical activation evoked by visual mental imagery as measured by fMRI. NeuroReport 11 (18), 3957-3962. https://doi.org/10.1097/00001756-200012180-00011Kosmyna, N., Lindgren, J. T., Lecuyer, A., Sep. 2018. Attending to visual stimuli versus performing visual imagery as a control strategy for EEG-based brain-computer interfaces. Scientific Reports 8 (1). https://doi.org/10.1038/s41598-018-31472-9Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., Arnaldi, B., Jan. 2007. A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering 4 (2), R1-R13. https://doi.org/10.1088/1741-2560/4/2/R01McCall, J., Dec. 2005. Genetic algorithms for modelling and optimisation. Journal of Computational and Applied Mathematics 184 (1), 205-222. https://doi.org/10.1016/j.cam.2004.07.034Melnik, A., Legkov, P., Izdebski, K., Karcher, S. M., Hairston, W. D., Ferris, D. P., Konig, P., Mar. 2017. Systems, subjects, sessions: To what extent do these factors influence EEG data? Frontiers in Human Neuroscience 11. https://doi.org/10.3389/fnhum.2017.00150Pedroni, A., Bahreini, A., Langer, N., Nov. 2019. Automagic: Standardized preprocessing of big EEG data. bioRxiv. https://doi.org/10.1101/460469Planelles, D., Hortal, E., Costa, Á., Úbeda, A., Iáez, E., Azorín, J., Sep. 2014. Evaluating classifiers to detect arm movement intention from EEG signals. Sensors 14 (10), 18172-18186. https://doi.org/10.3390/s141018172Proakis, John, M., Dimitri, 1996. Digital signal processing: principles, algorithms and applications. Prentice-Hall, 910-913.Reza, Abiri, S., Borhani, E. W., Sellers, Y., Jiang, X., Zhao, 2018. A comprehensive review of eeg-based brain-computer interface paradigms. Journal of Neural Engineering 16. https://doi.org/10.1088/1741-2552/aaf12eSeeck, M., Koessler, L., Bast, T., Leijten, F., Michel, C., Baumgartner, C., He, B., Beniczky, S., Oct. 2017. The standardized EEG electrode array of the IFCN. Clinical Neurophysiology 128 (10), 2070-2077. https://doi.org/10.1016/j.clinph.2017.06.254Swets, J., 2014. Signal detection theory and ROC analysis in psychology and diagnostics : collected papers. Psychology Press, New York, New York Hove, England. https://doi.org/10.4324/9781315806167Xie, S., Kaiser, D., Cichy, R. M., Aug. 2020. Visual imagery and perception share neural representations in the alpha frequency band. Current Biology 30 (15), 3062. https://doi.org/10.1016/j.cub.2020.07.023Zhang, W., Tan, C., Sun, F., Wu, H., Zhang, B., Dec. 2018. A review of EEGbased brain-computer interface systems design. Brain Science Advances 4 (2), 156-167. https://doi.org/10.26599/BSA.2018.905001