A new paradigm for BCI research

A new control paradigm for Brain Computer Interfaces (BCIs) is proposed. BCIs provide a means of communication direct from the brain to a computer that allows individuals with motor disabilities an additional channel of communication and control of their external environment. Traditional BCI control paradigms use motor imagery, frequency rhythm modification or the Event Related Potential (ERP) as a means of extracting a control signal. A new control paradigm for BCIs based on speech imagery is initially proposed. Further to this a unique system for identifying correlations between components of the EEG and target events is proposed and introduced

N400-like potentials and reaction times index semantic relations between highly repeated individual words

The N400 ERP is an electrophysiological index of semantic processing. Its amplitude varies with the semantic category of words, their concreteness, or whether their meaning matches that of a preceding context. The results of a number of studies suggest that these effects could be markedly reduced or suppressed for stimuli that are repeated. Nevertheless, we have recently shown that significant effects of semantic matching and category could be obtained on N400-like potentials elicited by massively repeated target words in a prime–target semantic categorization task. If such effects could be obtained when primes also are repeated, it would then be possible to study the semantic associations between individual words. The present study thus aimed to test this hypothesis while (1) controlling for a potential contribution of physical matching to the processing of repeated targets and (2) testing if the N400-like effects obtained in these conditions are modulated by task instruction, as are classic N400 effects. Two category words were used as primes and two exemplars as targets. In one block of trials, subjects had to respond according to the semantic relation between prime and target (semantic instruction) and, in another block, they had to report changes in letter case (physical instruction). Results showed that the amplitude of the N400-like ERP obtained was modulated by semantic matching and category but not by letter case. The effect of semantic matching was observed only in the semantic instruction block. Interestingly, the effect of category was not modulated by task instruction. An independent component analysis showed that the component that made the greatest contribution to the effect of semantic matching in the time window of the N400-like potential had a scalp distribution similar to that reported for the N400 and was best fit as a bilateral generator in the superior temporal gyrus. The use of repetition could thus allow, at least in explicit semantic tasks, a drastic simplification of N400 protocols. Highly repeated individual words could be used to study semantic relations between individual concepts

A psychometric measure of working memory capacity for configured body movement.

Working memory (WM) models have traditionally assumed at least two domain-specific storage systems for verbal and visuo-spatial information. We review data that suggest the existence of an additional slave system devoted to the temporary storage of body movements, and present a novel instrument for its assessment: the movement span task. The movement span task assesses individuals' ability to remember and reproduce meaningless configurations of the body. During the encoding phase of a trial, participants watch short videos of meaningless movements presented in sets varying in size from one to five items. Immediately after encoding, they are prompted to reenact as many items as possible. The movement span task was administered to 90 participants along with standard tests of verbal WM, visuo-spatial WM, and a gesture classification test in which participants judged whether a speaker's gestures were congruent or incongruent with his accompanying speech. Performance on the gesture classification task was not related to standard measures of verbal or visuo-spatial working memory capacity, but was predicted by scores on the movement span task. Results suggest the movement span task can serve as an assessment of individual differences in WM capacity for body-centric information

A P300 Based Cognitive Assessment Battery for Severely Motor-impaired and Overtly Non-responsive Patients

Diagnosing disorders of consciousness (DOC) is notoriously difficult, with estimates of misdiagnosis rates as high as 40%. Moreover, recent studies have demonstrated that patients who do not show signs of volitional motor responses can exhibit preserved command following detected by functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although these patients clearly retain some cognitive abilities, lack of consistent motor responses makes administration of standard neuropsychological tests impossible. Consequently, the extent of their cognitive function is unknown. In the current study, we developed and validated a P300b event related potential (ERP) neuropsychological battery in healthy participants to assess components of executive function without requiring motor output. First, participants were instructed to attend to a target auditory stimulus. P300b responses to attended relative to unattended stimuli were used as a neural proxy for detecting command following. To assess working memory capacity we adapted a digit span test to use a similar P300b response mechanism. Finally, reasoning was assessed by adapting a verbal reasoning task in the same manner. At the group level, and in a large majority of participants at the single-participant level, accurate performance could be detected using the P300b ERP, validating the potential utility of the battery. Additionally, the normalized magnitude of the P300b predicted individual differences in performance, but only when a suitable level of variability between participants was present. A post hoc Monte Carlo analysis was conducted to examine the necessary time required to conduct the battery as well as the interaction between time and performance in determining statistically significant performance. At 100% accuracy, a mean time of five minutes was required to achieve a significant result, with time increasing as a function of decreasing performance. These results demonstrate that covert control of attention, as measured by the P300b ERP, can be used to assess command following, working memory and reasoning abilities with a high degree of reliabilit

Predictive analysis of auditory attention from physiological signals

In recent years, there has been considerable interest in recording physiological signals from the human body to investigate various responses. Attention is one of the key aspects that physiologists, neuroscientists, and engineers have been exploring. Many theories have been established on auditory and visual selective attention. To date, the number of studies investigating the physiological responses of the human body to auditory attention on natural speech is, surprisingly, very limited, and there is a lack of public datasets. Investigating such physiological responses can open the door to new opportunities, as auditory attention plays a key role in many cognitive functionalities, thus impacting on learning and general task performance. In this thesis, we investigated auditory attention on the natural speech by processing physiological signals such as Electroencephalogram (EEG), Galvanic Skin Response (GSR), and Photoplethysmogram (PPG). An experiment was designed based on the well established dichotic listening task. In the experiment, we presented an audio stimulus under different auditory conditions: background noise level, length, and semanticity of the audio message. The experiment was conducted with 25 healthy, non-native speakers. The attention score was computed by counting the number of correctly identified words in the transcribed text response. All the physiological signals were labeled with their auditory condition and attention score. We formulated four predictive tasks exploiting the collected signals: Attention score, Noise level, Semanticity, and LWR (Listening, Writing, Resting, i.e., the state of the participant). In the first part, we analysed all the user text responses collected in the experiment. The statistical analysis reveals a strong dependency of the attention level on the auditory conditions. By applying hierarchical clustering, we could identify the experimental conditions that have similar effects on attention score. Significantly, the effect of semanticity appeared to vanish under high background noise. Then, analysing the signals, we found that the-state-of-the-art algorithms for artifact removal were inefficient for large datasets, as they require manual intervention. Thus, we introduced an EEG artifact removal algorithm with tuning parameters based on Wavelet Packet Decomposition (WPD). The proposed algorithm operates with two tuning parameters and three modes of wavelet filtering: Elimination, Linear Attenuation, and Soft-thresholding. Evaluating the algorithm performance, we observed that it outperforms state-of-the-art algorithms based on Independent Component Analysis (ICA). The evaluation was based on the spectrum, correlation, and distribution of the signals along with the performance in predictive tasks. We also demonstrate that a proper tuning of the algorithm parameters allows achieving further better results. After applying the artifact removal algorithm on EEG, we analysed the signals in terms of correlation of spectral bands of each electrode and attention score, semanticity, noise level, and state of the participant LWR). Next, we analyse the Event-Related Potential (ERP) on Listening, Writing and Resting segments of EEG signal, in addition to spectral analysis of GSR and PPG. With this thesis, we release the collected experimental dataset in the public domain, in order for the scientific community to further investigate the various auditory processing phenomena and their relation with EEG, GSR and PPG responses. The dataset can be used also to improve predictive tasks or design novel Brain-Computer-Interface (BCI) systems based on auditory attention. We also use the deeplearning approach to exploit the spatial relationship of EEG electrodes and inter-subject dependency of a model. As a domain application, we finally discuss the implications of auditory attention assessment for serious games and propose a 3-dimensional difficulty model to design game levels and dynamically adapt the difficulty to the player status

Relating EEG to continuous speech using deep neural networks: a review

Objective. When a person listens to continuous speech, a corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used to relate the EEG recording to the corresponding speech signal. The ability of linear models to find a mapping between these two signals is used as a measure of neural tracking of speech. Such models are limited as they assume linearity in the EEG-speech relationship, which omits the nonlinear dynamics of the brain. As an alternative, deep learning models have recently been used to relate EEG to continuous speech, especially in auditory attention decoding (AAD) and single-speech-source paradigms. Approach. This paper reviews and comments on deep-learning-based studies that relate EEG to continuous speech in AAD and single-speech-source paradigms. We point out recurrent methodological pitfalls and the need for a standard benchmark of model analysis. Main results. We gathered 29 studies. The main methodological issues we found are biased cross-validations, data leakage leading to over-fitted models, or disproportionate data size compared to the model's complexity. In addition, we address requirements for a standard benchmark model analysis, such as public datasets, common evaluation metrics, and good practices for the match-mismatch task. Significance. We are the first to present a review paper summarizing the main deep-learning-based studies that relate EEG to speech while addressing methodological pitfalls and important considerations for this newly expanding field. Our study is particularly relevant given the growing application of deep learning in EEG-speech decoding

Anesthetic-induced unresponsiveness: Electroencephalographic correlates and subjective experiences

Anesthetic drugs can induce reversible alterations in responsiveness, connectedness and consciousness. The measures based on electroencephalogram (EEG) have marked potential for monitoring the anesthetized state because of their relatively easy use in the operating room. In this study, 79 healthy young men participated in an awake experiment, and 47 participants continued to an anesthesia experiment where they received either dexmedetomidine or propofol as target-controlled infusion with stepwise increments until the loss of responsiveness. The participants were roused during the constant drug infusion and interviewed. The drug dose was increased to 1.5-fold to achieve a deeper unresponsive state. After regaining responsiveness, the participants were interviewed. EEG was measured throughout the experiment and the N400 event-related potential component and functional and directed connectivity were studied. Prefrontal-frontal connectivity in the alpha frequency band discriminated the states that differed with respect to responsiveness or drug concentration. The net direction of connectivity was frontal-to-prefrontal during unresponsiveness and reversed back to prefrontal-to-frontal upon return of responsiveness. The understanding of the meaning of spoken language, as measured with the N400 effect, was lost along with responsiveness but, in the dexmedetomidine group, the N400 component was preserved suggesting partial preservation of the processing of words during anesthetic-induced unresponsiveness. However, the N400 effect could not be detected in all the awake participants and the choice of analysis method had marked impact on its detection rate at the individual-level. Subjective experiences were common during unresponsiveness induced by dexmedetomidine and propofol but the experiences most often suggested disconnectedness from the environment. In conclusion, the doses of dexmedetomidine or propofol minimally sufficient to induce unresponsiveness do not render the participants unconscious and dexmedetomidine does not completely abolish the processing of semantic stimuli. The local anterior EEG connectivity in the alpha frequency band may have potential in monitoring the depth of dexmedetomidine- and propofol-induced anesthesia.Anesteettien aiheuttama vastauskyvyttömyys: aivosähkökäyräpohjaiset korrelaatit ja subjektiiviset kokemukset Anestesialääkkeillä voidaan saada aikaan palautuvia muutoksia vastauskykyisyydessä, kytkeytyneisyydessä ja tajunnassa. Aivosähkökäyrään (EEG) pohjautuvat menetelmät tarjoavat lupaavia mahdollisuuksia mitata anestesian vaikutusta aivoissa, sillä niitä on suhteellisen helppo käyttää leikkaussalissa. Tässä tutkimuksessa 79 tervettä nuorta miestä osallistui valvekokeeseen ja 47 heistä jatkoi anestesiakokeeseen. Anestesiakokeessa koehenkilöille annettiin joko deksmedetomidiinia tai propofolia tavoiteohjattuna infuusiona nousevia annosportaita käyttäen, kunnes he menettivät vastauskykynsä. Koehenkilöt herätettiin tasaisen lääkeinfuusion aikana ja haastateltiin. Koko kokeen ajan mitattiin EEG:tä, josta tutkittiin N400-herätevastetta sekä toiminnallista ja suunnattua konnektiivisuutta. Prefrontaali-frontaalivälillä mitattu konnektiivisuus alfa-taajuuskaistassa erotteli toisistaan tilat, jotka erosivat vastauskykyisyyden tai lääkepitoisuuden suhteen. Konnektiivisuuden vallitseva suunta oli frontaalialueilta prefrontaalialueille vastauskyvyttömyyden aikana, mutta se kääntyi takaisin prefrontaalisesta frontaaliseen kulkevaksi koehenkilöiden vastauskyvyn palatessa. N400-efektillä mitattu puhutun kielen ymmärtäminen katosi vastauskyvyn menettämisen myötä. Deksmedetomidiiniryhmässä N400-komponentti säilyi, mikä viittaa siihen, että anesteettien aiheuttaman vastauskyvyttömyyden aikana sanojen prosessointi voi säilyä osittain. Yksilötasolla N400-efektiä ei kuitenkaan havaittu edes kaikilla hereillä olevilla henkilöillä, ja analyysimenetelmän valinnalla oli suuri vaikutus herätevasteen havaitsemiseen. Subjektiiviset kokemukset olivat yleisiä deksmedetomidiinin ja propofolin aiheuttaman vastauskyvyttömyyden aikana, mutta kokemukset olivat usein ympäristöstä irtikytkeytyneitä. Yhteenvetona voidaan todeta, että deksmedetomidiini- ja propofoliannokset, jotka juuri ja juuri riittävät aikaansaamaan vastauskyvyttömyyden, eivät aiheuta tajuttomuutta. Deksmedetomidiini ei myöskään täysin estä merkityssisällöllisten ärsykkeiden käsittelyä. Frontaalialueen sisällä EEG:llä mitattu konnektiivisuus alfataajuuskaistassa saattaa olla tulevaisuudessa hyödyllinen menetelmä deksmedetomidiini- ja propofolianestesian syvyyden mittaamiseksi

Measuring Cross-Linguistic Influence in First- and Second-Generation Bilinguals: ERP vs. Acceptability Judgments

Two types of Spanish-English bilinguals were tested in an event-related potential (ERP) experiment on a contrast in the two languages exemplified in (1) and (2) in order to investigate linguistic permeability during processing of Spanish (1a and 2a). In Spanish, but not English, absence of the complementizer que is ungrammatical. (1) a. Qué hermana confesó Inés que había comido la tarta? b. *What sister did Inés confess that had eaten the cake? (2) a. *Qué hermana confesó Inés Ø había comido la tarta? b. What sister did Inés confess Ø had eaten the cake? In a first analysis, we grouped subjects by generation and compared ERP responses to que-less vs. que-full sentences. A significant N400 effect was found for first-, but not second-generation, suggesting reduced sensitivity to missing que for the latter. However, a second analysis, using linear mixed modeling to test predictiveness of individual speaker variables revealed generation to be non-predictive of N400 amplitude. Instead, current language use, cumulative exposure to English, and socioeconomic status (SES) were significant predictors for all subjects: increased English use, exposure, and SES resulted in smaller N400 amplitude to the anomaly in Spanish shown in (2a). Our results show that a priori classification of bilinguals masks gradient cross-linguistic effects, and processing is permeable in all bilinguals depending on amount of language use. Results from an acceptability judgment task administered to the same subjects using a subset of the same stimuli show that both subject groups judge que-less and que-full to be equally natural. These results suggest that behavioral measures that rely on metalinguistic judgments may not be good indicators of processing, and that having to appeal to metalinguistic knowledge may mask intrinsic knowledge