On the Relative Contribution of Deep Convolutional Neural Networks for SSVEP-based Bio-Signal Decoding in BCI Speller Applications

Brain-computer interfaces (BCI) harnessing Steady State Visual Evoked Potentials (SSVEP) manipulate the frequency and phase of visual stimuli to generate predictable oscillations in neural activity. For BCI spellers, oscillations are matched with alphanumeric characters allowing users to select target numbers and letters. Advances in BCI spellers can, in part, be accredited to subject-speci?c optimization, including; 1) custom electrode arrangements, 2) ?lter sub-band assessments and 3) stimulus parameter tuning. Here we apply deep convolutional neural networks (DCNN) demonstrating cross-subject functionality for the classi?cation of frequency and phase encoded SSVEP. Electroencephalogram (EEG) data are collected and classi?ed using the same parameters across subjects. Subjects ?xate forty randomly cued ?ickering characters (5 ×8 keyboard array) during concurrent wet-EEG acquisition. These data are provided by an open source SSVEP dataset. Our proposed DCNN, PodNet, achieves 86% and 77% of?ine Accuracy of Classi?cation across-subjects for two data capture periods, respectively, 6-seconds (information transfer rate= 40bpm) and 2-seconds (information transfer rate= 101bpm). Subjects demonstrating sub-optimal (< 70%) performance are classi?ed to similar levels after a short subject-speci?c training period. PodNet outperforms ?lter-bank canonical correlation analysis (FBCCA) for a low volume (3channel) clinically feasible occipital electrode con?guration. The networks de?ned in this study achieve functional performance for the largest number of SSVEP classes decoded via DCNN to date. Our results demonstrate PodNet achieves cross-subject, calibrationless classi?cation and adaptability to sub-optimal subject data and low-volume EEG electrode arrangements

Robust CNN architecture for classification of reach and grasp actions from neural correlates: an edge device perspective

Brain-computer interfaces (BCIs) systems traditionally use machine learning (ML) algorithms that require extensive signal processing and feature extraction. Deep learning (DL)-based convolutional neural networks (CNNs) recently achieved state-of-the-art electroencephalogram (EEG) signal classification accuracy. CNN models are complex and computationally intensive, making them difficult to port to edge devices for mobile and efficient BCI systems. For addressing the problem, a lightweight CNN architecture for efficient EEG signal classification is proposed. In the proposed model, a combination of a convolution layer for spatial feature extraction from the signal and a separable convolution layer to extract spatial features from each channel. For evaluation, the performance of the proposed model along with the other three models from the literature referred to as EEGNet, DeepConvNet, and EffNet on two different embedded devices, the Nvidia Jetson Xavier NX and Jetson Nano. The results of the Multivariant 2-way ANOVA (MANOVA) show a significant difference between the accuracies of ML and the proposed model. In a comparison of DL models, the proposed models, EEGNet, DeepConvNet, and EffNet, achieved 92.44 ± 4.30, 90.76 ± 4.06, 92.89 ± 4.23, and 81.69 ± 4.22 average accuracy with standard deviation, respectively. In terms of inference time, the proposed model performs better as compared to other models on both the Nvidia Jetson Xavier NX and Jetson Nano, achieving 1.9 sec and 16.1 sec, respectively. In the case of power consumption, the proposed model shows significant values on MANOVA (p < 0.05) on Jetson Nano and Xavier. Results show that the proposed model provides improved classification results with less power consumption and inference time on embedded platforms

Signal Processing Using Non-invasive Physiological Sensors

Non-invasive biomedical sensors for monitoring physiological parameters from the human body for potential future therapies and healthcare solutions. Today, a critical factor in providing a cost-effective healthcare system is improving patients' quality of life and mobility, which can be achieved by developing non-invasive sensor systems, which can then be deployed in point of care, used at home or integrated into wearable devices for long-term data collection. Another factor that plays an integral part in a cost-effective healthcare system is the signal processing of the data recorded with non-invasive biomedical sensors. In this book, we aimed to attract researchers who are interested in the application of signal processing methods to different biomedical signals, such as an electroencephalogram (EEG), electromyogram (EMG), functional near-infrared spectroscopy (fNIRS), electrocardiogram (ECG), galvanic skin response, pulse oximetry, photoplethysmogram (PPG), etc. We encouraged new signal processing methods or the use of existing signal processing methods for its novel application in physiological signals to help healthcare providers make better decisions

Comment le sens est-il extrait de l'information visuelle ? Le système visuel exploré des catégories à la conscience

Comment le sens est-il extrait de l'information visuelle ? Cette thèse est focalisée sur la capacité du système visuel d'humains et de singes à extraire et représenter l'information visuelle sur différents niveaux de complexité. Nous avons étudié différent niveaux de représentations visuelles, de la production de représentations visuelles primaires jusqu'à l'élaboration de représentations visuelles conscientes. Ce manuscrit présente six travaux dans lesquels nous avons exploré : (1) les attributs visuels nécessaires pour réaliser la tâche de catégorisation ultra rapide chez l'homme et le singe au moyen de méthodes psychophysiques, (2) la dynamique spatio-temporelle de l'attention visuelle chez l'homme au moyen de méthodes psychophysiques, (3) les corrélats neuronaux des représentations de haut niveau en EEG grâce au développement d'une nouvelle technique appelée SWIFT, (4) les corrélats neuronaux de la conscience visuelle dans la rivalité binoculaire en EEG, (5) la synchronie des signaux cérébraux en fonction de la reconnaissance consciente au moyen d'enregistrements intracrâniens chez des patients épileptiques et (6) les corrélats neuronaux associés à la prise de conscience chez le singe au moyen d'enregistrements intracrâniens. Les résultats de ces travaux nous ont permis d'ébaucher un modèle de la perception visuelle cherchant à dissocier l'attention et la conscience.How does sense emerges in the visual system? In this thesis we will be focused on the visual system of human and non-human primates and their large capacity of extract and represent visual information. We studied several levels of visual representations from those related to the extraction of coarse visual features to the emergence of conscious visual representations. This manuscript presents six works in which we explored: (1) the visual features necessary to perform ultra-rapid visual categorization in monkeys and humans using psychophysics, (2) the spatio-temporal dynamics of visual attention in humans using psychophysics, (3) the neural correlates of high-level visual representations using EEG tanks to the development of an innovative technique called SWIFT, (4) the neural correlates of visual consciousness under binocular rivalry using EEG, (5) the synchrony of brain signals as a function of conscious recognition using intracranial electrodes implanted in epileptic patients and (6) the neural correlates associated with conscious perception in monkeys using intracranial electrodes. The results of these works allowed outlining a tentative model of visual perception aimed to dissociate attention and consciousness

Design and Evaluation of a Hardware System for Online Signal Processing within Mobile Brain-Computer Interfaces

Brain-Computer Interfaces (BCIs) sind innovative Systeme, die eine direkte Kommunikation zwischen dem Gehirn und externen Geräten ermöglichen. Diese Schnittstellen haben sich zu einer transformativen Lösung nicht nur für Menschen mit neurologischen Verletzungen entwickelt, sondern auch für ein breiteres Spektrum von Menschen, das sowohl medizinische als auch nicht-medizinische Anwendungen umfasst. In der Vergangenheit hat die Herausforderung, dass neurologische Verletzungen nach einer anfänglichen Erholungsphase statisch bleiben, die Forscher dazu veranlasst, innovative Wege zu beschreiten. Seit den 1970er Jahren stehen BCIs an vorderster Front dieser Bemühungen. Mit den Fortschritten in der Forschung haben sich die BCI-Anwendungen erweitert und zeigen ein großes Potenzial für eine Vielzahl von Anwendungen, auch für weniger stark eingeschränkte (zum Beispiel im Kontext von Hörelektronik) sowie völlig gesunde Menschen (zum Beispiel in der Unterhaltungsindustrie). Die Zukunft der BCI-Forschung hängt jedoch auch von der Verfügbarkeit zuverlässiger BCI-Hardware ab, die den Einsatz in der realen Welt gewährleistet. Das im Rahmen dieser Arbeit konzipierte und implementierte CereBridge-System stellt einen bedeutenden Fortschritt in der Brain-Computer-Interface-Technologie dar, da es die gesamte Hardware zur Erfassung und Verarbeitung von EEG-Signalen in ein mobiles System integriert. Die Architektur der Verarbeitungshardware basiert auf einem FPGA mit einem ARM Cortex-M3 innerhalb eines heterogenen ICs, was Flexibilität und Effizienz bei der EEG-Signalverarbeitung gewährleistet. Der modulare Aufbau des Systems, bestehend aus drei einzelnen Boards, gewährleistet die Anpassbarkeit an unterschiedliche Anforderungen. Das komplette System wird an der Kopfhaut befestigt, kann autonom arbeiten, benötigt keine externe Interaktion und wiegt einschließlich der 16-Kanal-EEG-Sensoren nur ca. 56 g. Der Fokus liegt auf voller Mobilität. Das vorgeschlagene anpassbare Datenflusskonzept erleichtert die Untersuchung und nahtlose Integration von Algorithmen und erhöht die Flexibilität des Systems. Dies wird auch durch die Möglichkeit unterstrichen, verschiedene Algorithmen auf EEG-Daten anzuwenden, um unterschiedliche Anwendungsziele zu erreichen. High-Level Synthesis (HLS) wurde verwendet, um die Algorithmen auf das FPGA zu portieren, was den Algorithmenentwicklungsprozess beschleunigt und eine schnelle Implementierung von Algorithmusvarianten ermöglicht. Evaluierungen haben gezeigt, dass das CereBridge-System in der Lage ist, die gesamte Signalverarbeitungskette zu integrieren, die für verschiedene BCI-Anwendungen erforderlich ist. Darüber hinaus kann es mit einer Batterie von mehr als 31 Stunden Dauerbetrieb betrieben werden, was es zu einer praktikablen Lösung für mobile Langzeit-EEG-Aufzeichnungen und reale BCI-Studien macht. Im Vergleich zu bestehenden Forschungsplattformen bietet das CereBridge-System eine bisher unerreichte Leistungsfähigkeit und Ausstattung für ein mobiles BCI. Es erfüllt nicht nur die relevanten Anforderungen an ein mobiles BCI-System, sondern ebnet auch den Weg für eine schnelle Übertragung von Algorithmen aus dem Labor in reale Anwendungen. Im Wesentlichen liefert diese Arbeit einen umfassenden Entwurf für die Entwicklung und Implementierung eines hochmodernen mobilen EEG-basierten BCI-Systems und setzt damit einen neuen Standard für BCI-Hardware, die in der Praxis eingesetzt werden kann.Brain-Computer Interfaces (BCIs) are innovative systems that enable direct communication between the brain and external devices. These interfaces have emerged as a transformative solution not only for individuals with neurological injuries, but also for a broader range of individuals, encompassing both medical and non-medical applications. Historically, the challenge of neurological injury being static after an initial recovery phase has driven researchers to explore innovative avenues. Since the 1970s, BCIs have been at one forefront of these efforts. As research has progressed, BCI applications have expanded, showing potential in a wide range of applications, including those for less severely disabled (e.g. in the context of hearing aids) and completely healthy individuals (e.g. entertainment industry). However, the future of BCI research also depends on the availability of reliable BCI hardware to ensure real-world application. The CereBridge system designed and implemented in this work represents a significant leap forward in brain-computer interface technology by integrating all EEG signal acquisition and processing hardware into a mobile system. The processing hardware architecture is centered around an FPGA with an ARM Cortex-M3 within a heterogeneous IC, ensuring flexibility and efficiency in EEG signal processing. The modular design of the system, consisting of three individual boards, ensures adaptability to different requirements. With a focus on full mobility, the complete system is mounted on the scalp, can operate autonomously, requires no external interaction, and weighs approximately 56g, including 16 channel EEG sensors. The proposed customizable dataflow concept facilitates the exploration and seamless integration of algorithms, increasing the flexibility of the system. This is further underscored by the ability to apply different algorithms to recorded EEG data to meet different application goals. High-Level Synthesis (HLS) was used to port algorithms to the FPGA, accelerating the algorithm development process and facilitating rapid implementation of algorithm variants. Evaluations have shown that the CereBridge system is capable of integrating the complete signal processing chain required for various BCI applications. Furthermore, it can operate continuously for more than 31 hours with a 1800mAh battery, making it a viable solution for long-term mobile EEG recording and real-world BCI studies. Compared to existing research platforms, the CereBridge system offers unprecedented performance and features for a mobile BCI. It not only meets the relevant requirements for a mobile BCI system, but also paves the way for the rapid transition of algorithms from the laboratory to real-world applications. In essence, this work provides a comprehensive blueprint for the development and implementation of a state-of-the-art mobile EEG-based BCI system, setting a new benchmark in BCI hardware for real-world applicability

Deep learning for healthcare applications based on physiological signals: A review

Background and objective: We have cast the net into the ocean of knowledge to retrieve the latest scientific research on deep learning methods for physiological signals. We found 53 research papers on this topic, published from 01.01.2008 to 31.12.2017. Methods: An initial bibliometric analysis shows that the reviewed papers focused on Electromyogram(EMG), Electroencephalogram(EEG), Electrocardiogram(ECG), and Electrooculogram(EOG). These four categories were used to structure the subsequent content review. Results: During the content review, we understood that deep learning performs better for big and varied datasets than classic analysis and machine classification methods. Deep learning algorithms try to develop the model by using all the available input. Conclusions: This review paper depicts the application of various deep learning algorithms used till recently, but in future it will be used for more healthcare areas to improve the quality of diagnosi