Characterizing Motor System to Improve Training Protocols Used in Brain-Machine Interfaces Based on Motor Imagery

Motor imagery (MI)-based brain-machine interface (BMI) is a technology under development that actively modifies users’ perception and cognition through mental tasks, so as to decode their intentions from their neural oscillations, and thereby bringing some kind of activation. So far, MI as control task in BMIs has been seen as a skill that must be acquired, but neither user conditions nor controlled learning conditions have been taken into account. As motor system is a complex mechanism trained along lifetime, and MI-based BMI attempts to decode motor intentions from neural oscillations in order to put a device into action, motor mechanisms should be considered when prototyping BMI systems. It is hypothesized that the best way to acquire MI skills is following the same rules humans obey to move around the world. On this basis, new training paradigms consisting of ecological environments, identification of control tasks according to the ecological environment, transparent mapping, and multisensory feedback are proposed in this chapter. These new MI training paradigms take advantages of previous knowledge of users and facilitate the generation of mental image due to the automatic development of sensory predictions and motor behavior patterns in the brain. Furthermore, the effectuation of MI as an actual movement would make users feel that their mental images are being executed, and the resulting sensory feedback may allow forward model readjusting the imaginary movement in course

Using brain-computer interaction and multimodal virtual-reality for augmenting stroke neurorehabilitation

Every year millions of people suffer from stroke resulting to initial paralysis, slow motor recovery and chronic conditions that require continuous reha bilitation and therapy. The increasing socio-economical and psychological impact of stroke makes it necessary to find new approaches to minimize its sequels, as well as novel tools for effective, low cost and personalized reha bilitation. The integration of current ICT approaches and Virtual Reality (VR) training (based on exercise therapies) has shown significant improve ments. Moreover, recent studies have shown that through mental practice and neurofeedback the task performance is improved. To date, detailed in formation on which neurofeedback strategies lead to successful functional recovery is not available while very little is known about how to optimally utilize neurofeedback paradigms in stroke rehabilitation. Based on the cur rent limitations, the target of this project is to investigate and develop a novel upper-limb rehabilitation system with the use of novel ICT technolo gies including Brain-Computer Interfaces (BCI’s), and VR systems. Here, through a set of studies, we illustrate the design of the RehabNet frame work and its focus on integrative motor and cognitive therapy based on VR scenarios. Moreover, we broadened the inclusion criteria for low mobility pa tients, through the development of neurofeedback tools with the utilization of Brain-Computer Interfaces while investigating the effects of a brain-to-VR interaction.Todos os anos, milho˜es de pessoas sofrem de AVC, resultando em paral isia inicial, recupera¸ca˜o motora lenta e condic¸˜oes cr´onicas que requerem re abilita¸ca˜o e terapia cont´ınuas. O impacto socioecon´omico e psicol´ogico do AVC torna premente encontrar novas abordagens para minimizar as seque las decorrentes, bem como desenvolver ferramentas de reabilita¸ca˜o, efetivas, de baixo custo e personalizadas. A integra¸c˜ao das atuais abordagens das Tecnologias da Informa¸ca˜o e da Comunica¸ca˜o (TIC) e treino com Realidade Virtual (RV), com base em terapias por exerc´ıcios, tem mostrado melhorias significativas. Estudos recentes mostram, ainda, que a performance nas tare fas ´e melhorada atrav´es da pra´tica mental e do neurofeedback. At´e a` data, na˜o existem informac¸˜oes detalhadas sobre quais as estrat´egias de neurofeed back que levam a uma recupera¸ca˜o funcional bem-sucedida. De igual modo, pouco se sabe acerca de como utilizar, de forma otimizada, o paradigma de neurofeedback na recupera¸c˜ao de AVC. Face a tal, o objetivo deste projeto ´e investigar e desenvolver um novo sistema de reabilita¸ca˜o de membros supe riores, recorrendo ao uso de novas TIC, incluindo sistemas como a Interface C´erebro-Computador (ICC) e RV. Atrav´es de um conjunto de estudos, ilus tramos o design do framework RehabNet e o seu foco numa terapia motora e cognitiva, integrativa, baseada em cen´arios de RV. Adicionalmente, ampli amos os crit´erios de inclus˜ao para pacientes com baixa mobilidade, atrav´es do desenvolvimento de ferramentas de neurofeedback com a utilizac¸˜ao de ICC, ao mesmo que investigando os efeitos de uma interac¸˜ao c´erebro-para-RV

Die Wirksamkeit von Feedback und Trainingseffekten während der Alphaband Modulation über dem menschlichen sensomotorischen Cortex

Neural oscillations can be measured by electroencephalography (EEG) and these oscillations can be characterized by their frequency, amplitude and phase. The mechanistic properties of neural oscillations and their synchronization are able to explain various aspects of many cognitive functions such as motor control, memory, attention, information transfer across brain regions, segmentation of the sensory input and perception (Arnal and Giraud, 2012). The alpha band frequency is the dominant oscillation in the human brain. This oscillatory activity is found in the scalp EEG at frequencies around 8-13 Hz in all healthy adults (Makeig et al., 2002) and considerable interest has been generated in exploring EEG alpha oscillations with regard to their role in cognitive (Klimesch et al., 1993; Hanselmayr et al., 2005), sensorimotor (Birbaumer, 2006; Sauseng et al., 2009) and physiological (Lehmann, 1971; Niedermeyer, 1997; Kiyatkin, 2010) aspects of human life. The ability to voluntarily regulate the alpha amplitude can be learned with neurofeedback training and offers the possibility to control a brain-computer interface (BCI), a muscle independent interaction channel. BCI research is predominantly focused on the signal processing, the classification and the algorithms necessary to translate brain signals into control commands than on the person interacting with the technical system. The end-user must be properly trained to be able to successfully use the BCI and factors such as task instructions, training, and especially feedback can therefore play an important role in learning to control a BCI (Neumann and Kübler, 2003; Pfurtscheller et al., 2006, 2007; Allison and Neuper, 2010; Friedrich et al., 2012; Kaufmann et al., 2013; Lotte et al., 2013). The main purpose of this thesis was to investigate how end-users can efficiently be trained to perform alpha band modulation recorded over their sensorimotor cortex. The herein presented work comprises three studies with healthy participants and participants with schizophrenia focusing on the effects of feedback and training time on cortical activation patterns and performance. In the first study, the application of a realistic visual feedback to support end-users in developing a concrete feeling of kinesthetic motor imagery was tested in 2D and 3D visualization modality during a single training session. Participants were able to elicit the typical event-related desynchronisation responses over sensorimotor cortex in both conditions but the most significant decrease in the alpha band power was obtained following the three-dimensional realistic visualization. The second study strengthen the hypothesis that an enriched visual feedback with information about the quality of the input signal supports an easier approach for motor imagery based BCI control and can help to enhance performance. Significantly better performance levels were measurable during five online training sessions in the groups with enriched feedback as compared to a conventional simple visual feedback group, without significant differences in performance between the unimodal (visual) and multimodal (auditory–visual) feedback modality. Furthermore, the last study, in which people with schizophrenia participated in multiple sessions with simple feedback, demonstrated that these patients can learn to voluntarily regulate their alpha band. Compared to the healthy group they required longer training times and could not achieve performance levels as high as the control group. Nonetheless, alpha neurofeedback training lead to a constant increase of the alpha resting power across all 20 training session. To date only little is known about the effects of feedback and training time on BCI performance and cortical activation patterns. The presented work contributes to the evidence that healthy individuals can benefit from enriched feedback: A realistic presentation can support participants in getting a concrete feeling of motor imagery and enriched feedback, which instructs participants about the quality of their input signal can give support while learning to control the BCI. This thesis demonstrates that people with schizophrenia can learn to gain control of their alpha oscillations recorded over the sensorimotor cortex when participating in sufficient training sessions. In conclusion, this thesis improved current motor imagery BCI feedback protocols and enhanced our understanding of the interplay between feedback and BCI performance.Die Wirksamkeit von Feedback und Trainingseffekten während der Alphaband Modulation über dem menschlichen sensomotorischen Corte

Proceedings of the 3rd International Mobile Brain/Body Imaging Conference : Berlin, July 12th to July 14th 2018

The 3rd International Mobile Brain/Body Imaging (MoBI) conference in Berlin 2018 brought together researchers from various disciplines interested in understanding the human brain in its natural environment and during active behavior. MoBI is a new imaging modality, employing mobile brain imaging methods like the electroencephalogram (EEG) or near infrared spectroscopy (NIRS) synchronized to motion capture and other data streams to investigate brain activity while participants actively move in and interact with their environment. Mobile Brain / Body Imaging allows to investigate brain dynamics accompanying more natural cognitive and affective processes as it allows the human to interact with the environment without restriction regarding physical movement. Overcoming the movement restrictions of established imaging modalities like functional magnetic resonance tomography (MRI), MoBI can provide new insights into the human brain function in mobile participants. This imaging approach will lead to new insights into the brain functions underlying active behavior and the impact of behavior on brain dynamics and vice versa, it can be used for the development of more robust human-machine interfaces as well as state assessment in mobile humans.DFG, GR2627/10-1, 3rd International MoBI Conference 201

Enhancing brain-computer interfacing through advanced independent component analysis techniques

A Brain-computer interface (BCI) is a direct communication system between a brain and an external device in which messages or commands sent by an individual do not pass through the brain’s normal output pathways but is detected through brain signals. Some severe motor impairments, such as Amyothrophic Lateral Sclerosis, head trauma, spinal injuries and other diseases may cause the patients to lose their muscle control and become unable to communicate with the outside environment. Currently no effective cure or treatment has yet been found for these diseases. Therefore using a BCI system to rebuild the communication pathway becomes a possible alternative solution. Among different types of BCIs, an electroencephalogram (EEG) based BCI is becoming a popular system due to EEG’s fine temporal resolution, ease of use, portability and low set-up cost. However EEG’s susceptibility to noise is a major issue to develop a robust BCI. Signal processing techniques such as coherent averaging, filtering, FFT and AR modelling, etc. are used to reduce the noise and extract components of interest. However these methods process the data on the observed mixture domain which mixes components of interest and noise. Such a limitation means that extracted EEG signals possibly still contain the noise residue or coarsely that the removed noise also contains part of EEG signals embedded. Independent Component Analysis (ICA), a Blind Source Separation (BSS) technique, is able to extract relevant information within noisy signals and separate the fundamental sources into the independent components (ICs). The most common assumption of ICA method is that the source signals are unknown and statistically independent. Through this assumption, ICA is able to recover the source signals. Since the ICA concepts appeared in the fields of neural networks and signal processing in the 1980s, many ICA applications in telecommunications, biomedical data analysis, feature extraction, speech separation, time-series analysis and data mining have been reported in the literature. In this thesis several ICA techniques are proposed to optimize two major issues for BCI applications: reducing the recording time needed in order to speed up the signal processing and reducing the number of recording channels whilst improving the final classification performance or at least with it remaining the same as the current performance. These will make BCI a more practical prospect for everyday use. This thesis first defines BCI and the diverse BCI models based on different control patterns. After the general idea of ICA is introduced along with some modifications to ICA, several new ICA approaches are proposed. The practical work in this thesis starts with the preliminary analyses on the Southampton BCI pilot datasets starting with basic and then advanced signal processing techniques. The proposed ICA techniques are then presented using a multi-channel event related potential (ERP) based BCI. Next, the ICA algorithm is applied to a multi-channel spontaneous activity based BCI. The final ICA approach aims to examine the possibility of using ICA based on just one or a few channel recordings on an ERP based BCI. The novel ICA approaches for BCI systems presented in this thesis show that ICA is able to accurately and repeatedly extract the relevant information buried within noisy signals and the signal quality is enhanced so that even a simple classifier can achieve good classification accuracy. In the ERP based BCI application, after multichannel ICA the data just applied to eight averages/epochs can achieve 83.9% classification accuracy whilst the data by coherent averaging can reach only 32.3% accuracy. In the spontaneous activity based BCI, the use of the multi-channel ICA algorithm can effectively extract discriminatory information from two types of singletrial EEG data. The classification accuracy is improved by about 25%, on average, compared to the performance on the unpreprocessed data. The single channel ICA technique on the ERP based BCI produces much better results than results using the lowpass filter. Whereas the appropriate number of averages improves the signal to noise rate of P300 activities which helps to achieve a better classification. These advantages will lead to a reliable and practical BCI for use outside of the clinical laboratory

Functional Equivalence of Imagined vs. Real Performance of an Inhibitory Task: An EEG/ERP Study

Early neuroimaging and electrophysiological studies suggested that motor imagery recruited a different network than motor execution. However, several studies have provided evidence for the involvement of the same circuits in motor imagery tasks, in the absence of overt responses. The present study aimed to test whether imagined performance of a stop-signal task produces a similar pattern of motor-related EEG activity than that observed during real performance. To this end, mu and beta eventrelated desynchronization (ERD) and the Lateralized Readiness Potential (LRP) were analyzed. The study also aimed to clarify the functional significance of the Stop-N2 and Stop-P3 event-related potential (ERPs) components, which were also obtained during both real and imagined performance. The results showed a common pattern of brain electrical activity, and with a similar time course, during covert performance and overt execution of the stop-signal task: presence of LRP and Stop-P3 in the imagined condition and identical LRP onset, and similar mu and beta ERD temporal windows for both conditions. These findings suggest that a similar inhibitory network may be activated during both overt and covert execution of the task. Therefore, motor imagery may be useful to improve inhibitory skills and to develop new communicating systems for Brain-Computer Interface (BCI) devices based on inhibitory signalsThis research was funded by Spanish Ministerio de Economía y Competitividad (Reference PSI2013-43594-R). AJG-V was supported by a research grant from the Fundación Ramón DominguezS

Neurophysiological investigation of auditory augmentation to facilitate observational learning of everyday actions

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityThis thesis explores the neurophysiology of auditory augmentation of observed actions, and its effects on motor learning, neural activity, and plasticity. To this end, three studies were conducted. In the first study, we used Transcranial Magnetic Stimulation (TMS) to determine the effects of practising a motor task via sonification of combined action observation and motor imagery (sAOMI) on corticospinal excitability, compared to practising without extrinsic auditory information. In addition, we aimed at probing practice-dependent audiomotor plasticity. To this extent we used a variation of a commonly used method to probe and induce plasticity in humans, auditory paired associative stimulation (aPAS). Practice significantly increased corticospinal excitability, but sonification did not affect it. In addition, while aPAS completed alone significantly modulated corticospinal excitability, when practice primed aPAS, no neuromodulation was found. In a follow-up study, we explored the effects of sAOMI on corticospinal excitability during action observation (AO) or AOMI of the same action, and whether sonification induced audiomotor resonance, which is usually interpreted as development of an audiomotor association. The results corroborated and extended the findings of the previous study: practice increased corticospinal excitability at rest and during AO and AOMI, but sonification did not affect it. In addition, with sonification did not induce an audiomotor association. In a third study, we used electroencephalography (EEG) and other psychophysical measures, including a motor imagery questionnaire and mental chronometry, to assess changes resulting from practising with sonified action observation, followed by motor imagery of the same action. After two practice sessions, performance and kinaesthetic motor imagery vividness significantly improved, and participants’ mental chronometry was significantly more aligned with the speed of the observed action, compared to pre-practice measures. Sonification did not induce changes in any measure. EEG analysis revealed that participants who practised with sonification were able to sustain event-related desynchronization (ERD) in the lower alpha band (7-10 Hz) for longer, compared to participants who practices without sonification. No changes in higher alpha (10-12 Hz) or Beta (16-25 Hz) bands were found. Taken together, convergent results from this thesis suggest that sonified action observation has little effect on neurophysiological and behavioural markers of motor imagery ability and performance in healthy individuals. On the other hand, practising with sonified action observation may induce attentional modulations that enhance the learner’s ability to sustain action-related attentional processing for longer. We discuss these results in the context of contemporary neurocomputational theories of perception and action

EEG OSCILLATORY ACTIVITIES FROM HUMAN MOTOR BRAIN

Motor skills are essential in people’s daily life in exploring and interacting with the ambient environment. Impairments to motor functions affect the acquisition of motor skills, which not only reduce the quality of life, but also impose heavy economic burdens to sufferers and their families. Oscillatory activities in electroencephalography (EEG), such as the mu rhythm, present functional correlation to motor functions, which provide accessible windows to understand underlying neural mechanism in healthy persons and perform diagnoses in patients with various motor impairments. It is thus of significant importance to further investigate classic and/or identify new motor-related EEG oscillatory activities. In this dissertation, EEG oscillations from both infants and adults are investigated to uncover motor-related neural information noninvasively from the human brain regarding their developmental changes and movement representations of body parts, respectively. In typical developing infants at 5-7 months of age, knowledge about mu rhythm development is expanded by capturing subtle developmental changes of its characteristics in a fine age resolution, through the development of new spatio-spectral analysis of EEG data recorded longitudinally on a weekly basis. In adults, motor tasks involving fine body parts are studied to investigate EEG resolutions in decoding movements/motor imageries of individual fingers, which have only been addressed in large body parts in literature. Discriminative information in EEG oscillations about motor tasks of fine body parts is revealed through the discovery of a novel type of spectral structures in EEG, which exhibits better sensitivity to movements of fine body parts than the classic mu rhythm. The findings in this dissertation broaden the scope of neural information in EEG oscillations in relation to motor functions, and contribute to the understanding about human motor functions at various life stages. These results and technologies are promising to be translated to patient studies in the future

Development of Brain Machine Interface Systems and its Applications to Prosthetic Hand Control

東京電機大学201