Co-adaptive control strategies in assistive Brain-Machine Interfaces

A large number of people with severe motor disabilities cannot access any of the available control inputs of current assistive products, which typically rely on residual motor functions. These patients are therefore unable to fully benefit from existent assistive technologies, including communication interfaces and assistive robotics. In this context, electroencephalography-based Brain-Machine Interfaces (BMIs) offer a potential non-invasive solution to exploit a non-muscular channel for communication and control of assistive robotic devices, such as a wheelchair, a telepresence robot, or a neuroprosthesis. Still, non-invasive BMIs currently suffer from limitations, such as lack of precision, robustness and comfort, which prevent their practical implementation in assistive technologies. The goal of this PhD research is to produce scientific and technical developments to advance the state of the art of assistive interfaces and service robotics based on BMI paradigms. Two main research paths to the design of effective control strategies were considered in this project. The first one is the design of hybrid systems, based on the combination of the BMI together with gaze control, which is a long-lasting motor function in many paralyzed patients. Such approach allows to increase the degrees of freedom available for the control. The second approach consists in the inclusion of adaptive techniques into the BMI design. This allows to transform robotic tools and devices into active assistants able to co-evolve with the user, and learn new rules of behavior to solve tasks, rather than passively executing external commands. Following these strategies, the contributions of this work can be categorized based on the typology of mental signal exploited for the control. These include: 1) the use of active signals for the development and implementation of hybrid eyetracking and BMI control policies, for both communication and control of robotic systems; 2) the exploitation of passive mental processes to increase the adaptability of an autonomous controller to the user\u2019s intention and psychophysiological state, in a reinforcement learning framework; 3) the integration of brain active and passive control signals, to achieve adaptation within the BMI architecture at the level of feature extraction and classification

Brain-Computer Interface Based on Unsupervised Methods to Recognize Motor Intention for Command of a Robotic Exoskeleton

Stroke and road traffic injuries may severely affect movements of lower-limbs in humans, and consequently the locomotion, which plays an important role in daily activities, and the quality of life. Robotic exoskeleton is an emerging alternative, which may be used on patients with motor deficits in the lower extremities to provide motor rehabilitation and gait assistance. However, the effectiveness of robotic exoskeletons may be reduced by the autonomous ability of the robot to complete the movement without the patient involvement. Then, electroencephalography signals (EEG) have been addressed to design brain-computer interfaces (BCIs), in order to provide a communication pathway for patients perform a direct control on the exoskeleton using the motor intention, and thus increase their participation during the rehabilitation. Specially, activations related to motor planning may help to improve the close loop between user and exoskeleton, enhancing the cortical neuroplasticity. Motor planning begins before movement onset, thus, the training stage of BCIs may be affected by the intuitive labeling process, as it is not possible to use reference signals, such as goniometer or footswitch, to select those time periods really related to motor planning. Therefore, the gait planning recognition is a challenge, due to the high uncertainty of selected patterns, However, few BCIs based on unsupervised methods to recognize gait planning/stopping have been explored. This Doctoral Thesis presents unsupervised methods to improve the performance of BCIs during gait planning/ stopping recognition. At this context, an adaptive spatial filter for on-line processing based on the Concordance Correlation Coefficient (CCC) was addressed to preserve the useful information on EEG signals, while rejecting neighbor electrodes around the electrode of interest. Here, two methods for electrode selection were proposed. First, both standard deviation and CCC between target electrodes and their correspondent neighbor electrodes are analyzed on sliding windows to select those neighbors that are highly correlated. Second, Zscore analysis is performed to reject those neighbor electrodes whose amplitude values presented significant difference in relation to other neighbors. Furthermore, another method that uses the representation entropy and the maximal information compression index (MICI) was proposed for feature selection, which may be robust to select patterns, as only it depends on cluster distribution. In addition, a statistical analysis was introduced here to adjust, in the training stage of BCIs, regularized classifiers, such as support vector machine (SVM) and regularized discriminant analysis (RDA). Six subjects were adopted to evaluate the performance of different BCIs based on the proposed viii methods, during gait planning/stopping recognition. The unsupervised approach for feature selection showed similar performance to other methods based on linear discriminant analysis (LDA), when it was applied in a BCI based on the traditional Weighted Average to recognize gait planning. Additionally, the proposed adaptive filter improved the performance of BCIs based on traditional spatial filters, such as Local Average Reference (LAR) and WAR, as well as others BCIs based on powerful methods, such as Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP) and Riemannian kernel (RK). RK presented the best performance in comparison to CSP and FBCSP, which agrees with the hypothesis that unsupervised methods may be more appropriate to analyze clusters of high uncertainty, as those formed by motor planning. BCIs using adaptive filter based on Zscore analysis, with an unsupervised approach for feature selection and RDA showed promising results to recognize both gait planning and gait stopping, achieving for three subjects, good values of true positive rate (>70%) and false positive (<16%). Thus, the proposed methods may be used to obtain an optimized BCI that preserves the useful information, enhancing the gait planning/stopping recognition. In addition, the method for feature selection has low computational cost, which may be suitable for applications that demand short time of training, such as clinical application time

Advances in Robot Navigation

Robot navigation includes different interrelated activities such as perception - obtaining and interpreting sensory information; exploration - the strategy that guides the robot to select the next direction to go; mapping - the construction of a spatial representation by using the sensory information perceived; localization - the strategy to estimate the robot position within the spatial map; path planning - the strategy to find a path towards a goal location being optimal or not; and path execution, where motor actions are determined and adapted to environmental changes. This book integrates results from the research work of authors all over the world, addressing the abovementioned activities and analyzing the critical implications of dealing with dynamic environments. Different solutions providing adaptive navigation are taken from nature inspiration, and diverse applications are described in the context of an important field of study: social robotics

Effects of Interpretation Error on User Learning in Novel Input Mechanisms

Novel input mechanisms generate signals that are interpreted as commands in computer systems. Sometimes noise from various sources can cause the system to produce errors when attempting to interpret the signal, causing a misrepresentation of the user's intention. While research has been done in understanding how these interpretation errors affect the performance of users of novel signal-based input mechanisms, such as a brain-computer interface (BCI), there is a lack of knowledge in how user learning is affected. Previous literature in command-based selection tasks has suggested that errors will have a negative impact on expertise development; however, the presence of errors could conversely improve a user's learning by demanding more attention from the user. This thesis begins by studying people's ability to use a novel input mechanism with a noisy input signal: a motor imagery BCI. By converting a user's brain signals into computer commands, a user could complete selection tasks using imagined movement. However, the high degree of interpretation errors caused by noise in the input signals made it difficult to differentiate the user's intent from the noise. As such, the results of the BCI study served as motivation to test the effects of interpretation errors on user learning. Two studies were conducted to determine how user performance and learning were affected by different rates of interpretation error in a novel input mechanism. The results from these two studies showed that interpretation errors led to slower task completion times, lower accuracy in memory recall, greater rates of user errors, and increased frustration. This new knowledge about the effects of interpretation errors can contribute to better design of input mechanisms and training programs for novel input systems

A Brain-computer Interface Architecture Based On Motor Mental Tasks And Music Imagery

This present research proposes a Brain-Computer Interface (BCI) architecture adapted to motor mental tasks and music imagery. For that purpose the statistical properties of the electroencephalographic signal (EEG) were studied, such as its probability distribution function, stationarity, correlation and signal-to-noise ratio (SNR), in order to obtain a minimal empirical and well-founded parameter system for online classification. Stationarity tests were used to estimate the length of the time windows and a minimum length of 1.28 s was obtained. Four algorithms for artifact reduction were tested: threshold analysis, EEG filtering and two Independent Component Analysis (ICA) algorithms. This analysis concluded that the algorithm fastICA is suitable for online artifact removal. The feature extraction used the Power Spectral Density (PSD) and three methods were tested for automatic selection of features in order to have a training step independent of the mental task paradigm, with the best performance obtained with the Kullback-Leibler symmetric divergence method. For the classification, the Linear Discriminant Analysis (LDA) was used and a step of reclassification is suggested. A study of four motor mental tasks and a non-motor related mental task is performed by comparing their periodograms, Event-Related desynchronization/synchronization (ERD/ERS) and SNR. The mental tasks are the imagination of either movement of right and left hands, both feet, rotation of a cube and sound imagery. The EEG SNR was estimated by a comparison with the correlation between the ongoing average and the final ERD/ERS curve, in which we concluded that the mental task of sound imagery would need approximately five times more epochs than the motor-related mental tasks. The ERD/ERS could be measured even for frequencies near 100 Hz, but in absolute amplitudes, the energy variation at 100 Hz was one thousand times smaller than for 10 Hz, which implies that there is a small probability of online detection for BCI applications in high frequency. Thus, most of the usable information for online processing and BCIs corresponds to the &#945;/µ band (low frequency). Finally, the ERD/ERS scalp maps show that the main difference between the sound imagery task and the motor-related mentaltasks is the absence of ERD at the µ band, in the central electrodes, and the presence of ERD at the &#945;band in the temporal and lateral-frontal electrodes, which correspond tothe auditory cortex, the Wernickes area and the Brocas area

Mapping Sensorimotor Function and Controlling Upper Limb Neuroprosthetics with Electrocorticography

Electrocorticography (ECoG) occupies a unique intermediate niche between microelectrode recordings of single neurons and recordings of whole brain activity via functional magnetic resonance imaging (fMRI). ECoG’s combination of high temporal resolution and wide area coverage make it an ideal modality for both functional brain mapping and brain-machine interface (BMI) for control of prosthetic devices. This thesis demonstrates the utility of ECoG, particularly in high gamma frequencies (70-120 Hz), for passive online mapping of language and motor behaviors, online control of reaching and grasping of an advanced robotic upper limb, and mapping somatosensory digit representations in the postcentral gyrus. The dissertation begins with a brief discussion of the framework for neuroprosthetic control developed by the collaboration between Johns Hopkins and JHU Applied Physics Laboratory (JHU/APL). Second, the methodology behind an online spatial-temporal functional mapping (STFM) system is described. Trial-averaged spatiotemporal maps of high gamma activity were computed during a visual naming and a word reading task. The system output is subsequently shown and compared to stimulation mapping. Third, simultaneous and independent ECoG-based control of reaching and grasping is demonstrated with the Modular Prosthetic Limb (MPL). The STFM system was used to identify channels whose high gamma power significantly and selectively increases during either reaching or grasping. Using this technique, two patients were able to rapidly achieve naturalistic control over simple movements by the MPL. Next, high-density ECoG (hdECoG) was used to map the cortical responses to mechanical vibration of the fingertips. High gamma responses exhibited a strong yet overlapping somatotopy that was not well replicated in other frequency bands. These responses are strong enough to be detected in single trials and used to classify the finger being stimulated with over 98% accuracy. Finally, the role of ECoG is discussed for functional mapping and BMI applications. ECoG occupies a unique role among neural recording modalities as a tool for functional mapping, but must prove its value relative to stimulation mapping. For BMI, ECoG lags microelectrode arrays but hdECoG may provide a more robust long-term interface with optimal spacing for sampling relevant cortical representations

Actas de SABI2020

Los temas salientes incluyen un marcapasos pulmonar que promete complementar y eventualmente sustituir la conocida ventilación mecánica por presión positiva (intubación), el análisis de la marchaespontánea sin costosos equipamientos, las imágenes infrarrojas y la predicción de la salud cardiovascular en temprana edad por medio de la biomecánica arterial

Manipulador aéreo con brazos antropomórficos de articulaciones flexibles

[Resumen] Este artículo presenta el primer robot manipulador aéreo con dos brazos antropomórficos diseñado para aplicarse en tareas de inspección y mantenimiento en entornos industriales de difícil acceso para operarios humanos. El robot consiste en una plataforma aérea multirrotor equipada con dos brazos antropomórficos ultraligeros, así como el sistema de control integrado de la plataforma y los brazos. Una de las principales características del manipulador es la flexibilidad mecánica proporcionada en todas las articulaciones, lo que aumenta la seguridad en las interacciones físicas con el entorno y la protección del propio robot. Para ello se ha introducido un compacto y simple mecanismo de transmisión por muelle entre el eje del servo y el enlace de salida. La estructura en aluminio de los brazos ha sido cuidadosamente diseñada de forma que los actuadores estén aislados frente a cargas radiales y axiales que los puedan dañar. El manipulador desarrollado ha sido validado a través de experimentos en base fija y en pruebas de vuelo en exteriores.Ministerio de Economía y Competitividad; DPI2014-5983-C2-1-

Perception Of Bci Assistive Technology By Post-ischemic Stroke Patients

Stroke is the leading cause of acquired physical impairment in adult in the world. One person in six seconds is having a stroke somewhere in the world. It is estimated that one-third of stroke victims will be handicapped, and will require assistive technology of some sort. There are many BCI therapeutic resources in development that can be used for treating patients who have any physical challenge. The objective of this study is to explore the perception of BCI assistive technology by the post-stroke patients that have sequelae. We applied a home developed questionnaire and conducted a semi-structured interview by phone to explore the perception of patients towards BCI procedures. We studied seven post-ischemic stroke patients (4 men) with a mean age of 63 years (range 34-80 years). Six subjects had incomplete basic education and only one subject had completed high school. The median Rankin score was 3 (range 2-4). We found positive to very positive perception on the usage of BCI. A pleasant experience was described by all patients and no complaints were reported. Most subjects misinterpreted the research procedures regarding them as part of their treatment. In conclusion, the overall perception of BCI by the stroke patients was positive, and there is a willingness of trying this type of technology in particular when physicians are part of the BCI implementation process. © 2013 IEEE.Mackay, J., Mensah, A.G., (2004) The Atlas of Heart Disease and Stroke, , World Health OrganizationNicolas-Alonso, L.F., Gomez-Gil, J., Brain computer interfaces, a review (2012) Sensors, 12 (2), pp. 1211-1279Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M., Brain-computer interfaces for communication and control (2002) Clin. Neurophysiol., 113, pp. 767-791Guger, C., Harkam, W., Hertnaes, C., Pfurtscheller, G., Prosthetic control by an EEG-based brain-computer interface (BCI) (1999) Presented at the AAATE 5th Eur. Conf. Adv. Assistive Technol., , Düsseldorf, GermanyContreras, J.L., Vidal, A.P., Agashe, H., Paek, A., Restoration of whole body movement: Toward a noninvasive brain-machine interface system (2012) IEEE Pulse, 3 (1), pp. 34-37. , JanuaryFifer, M.S., Acharya, S., Benz, H.L., Mollazadeh, M., Crone, N.E., Thakor, N.V., Toward electrocortico graphic control of a dexterous upper limb prosthesis: Building brain-machine interfaces (2012) Pulse, IEEE, 3 (1), pp. 38-42. , JanBirbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Flor, H., A spelling device for the paralysed (1999) Nature, 398, pp. 297-298Li, H., Li, Y., Guan, C., An effective BCI speller based on semi supervised learning (2006) Engineering in Medicine and Biology Society, 2006. EMBS '06. 28th Annual International Conference of the IEEE, pp. 1161-1164. , Aug. 30 2006-Sept. 3Rebsamen, B., Burdet, E., Guan, C., Zhang, H., Teo, C.L., Zeng, Q., Laugier, C., Ang Jr., M.H., (2007) Controlling A Wheelchair Indoors Using Thought, 22, pp. 18-24Bastos, T.F., Muller, S.M.T., Benevides, A.B., Sarcinelli-Filho, M., Robotic wheelchair commanded by SSVEP, motor imagery and word generation (2011) Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp. 4753-4756. , Aug. 30 2011-Sept. 3Wolpaw, J.R., Brain-computer interfaces as new brain output pathways (2007) The Journal of Physiology, 579, pp. 613-619Lebedev, M.A., Nicolelis, M.A.L., Brain-machine interfaces: Past, present and future (2006) Trends in Neurosciences, 29 (9), pp. 536-546McFarland, D.J., Wolpaw, J.R., Brain-computer interface operation of robotic and prosthetic devices (2008) Computer, 41 (10), pp. 52-56. , OctGomez-Rodriguez, M., Grosse-Wentrup, M., Hill, J., Gharabaghi, A., Scholkopf, B., Peters, J., Towards brain-robot interfaces in stroke rehabilitation. Rehabilitation robotics (ICORR) (2011) 2011 IEEE International Conference on, pp. 1-6. , June 29 2011-JulyBuch, E., Weber, C., Cohen, L.G., Braun, C., Dimyan, M.A., Ard, T., Mellinger, J., Birbaumer, N., Think to move: A neuromagnetic brain-computer interface (BCI) system for chronic stroke (2008) Stroke, 39, pp. 910-917. , MarThe UK-TIA aspirin trial: Interim results (1988) Br Med J, 296, pp. 316-320. , UK-TIA Study Grou