Speech Processes for Brain-Computer Interfaces

Speech interfaces have become widely used and are integrated in many applications and devices. However, speech interfaces require the user to produce intelligible speech, which might be hindered by loud environments, concern to bother bystanders or the general in- ability to produce speech due to disabilities. Decoding a usera s imagined speech instead of actual speech would solve this problem. Such a Brain-Computer Interface (BCI) based on imagined speech would enable fast and natural communication without the need to actually speak out loud. These interfaces could provide a voice to otherwise mute people. This dissertation investigates BCIs based on speech processes using functional Near In- frared Spectroscopy (fNIRS) and Electrocorticography (ECoG), two brain activity imaging modalities on opposing ends of an invasiveness scale. Brain activity data have low signal- to-noise ratio and complex spatio-temporal and spectral coherence. To analyze these data, techniques from the areas of machine learning, neuroscience and Automatic Speech Recog- nition are combined in this dissertation to facilitate robust classification of detailed speech processes while simultaneously illustrating the underlying neural processes. fNIRS is an imaging modality based on cerebral blood flow. It only requires affordable hardware and can be set up within minutes in a day-to-day environment. Therefore, it is ideally suited for convenient user interfaces. However, the hemodynamic processes measured by fNIRS are slow in nature and the technology therefore offers poor temporal resolution. We investigate speech in fNIRS and demonstrate classification of speech processes for BCIs based on fNIRS. ECoG provides ideal signal properties by invasively measuring electrical potentials artifact- free directly on the brain surface. High spatial resolution and temporal resolution down to millisecond sampling provide localized information with accurate enough timing to capture the fast process underlying speech production. This dissertation presents the Brain-to- Text system, which harnesses automatic speech recognition technology to decode a textual representation of continuous speech from ECoG. This could allow to compose messages or to issue commands through a BCI. While the decoding of a textual representation is unparalleled for device control and typing, direct communication is even more natural if the full expressive power of speech - including emphasis and prosody - could be provided. For this purpose, a second system is presented, which directly synthesizes neural signals into audible speech, which could enable conversation with friends and family through a BCI. Up to now, both systems, the Brain-to-Text and synthesis system are operating on audibly produced speech. To bridge the gap to the final frontier of neural prostheses based on imagined speech processes, we investigate the differences between audibly produced and imagined speech and present first results towards BCI from imagined speech processes. This dissertation demonstrates the usage of speech processes as a paradigm for BCI for the first time. Speech processes offer a fast and natural interaction paradigm which will help patients and healthy users alike to communicate with computers and with friends and family efficiently through BCIs

Learning, Mood, and Music: Depression, anxiety, and stress reflect processing biases in positive and negative chord sequences

Cognitive biases in information processing of valenced stimuli are a major contributor to the phenomenology of mood disorders. However, current screening tools for mood disorders rely on self-report questionnaires, which include uncomfortably invasive questions and are confounded by socially desirable responding. Taken together, assessing information processing biases may be a promising proxy to screen non-invasively for mood disorders. Here, we report data of 60 participants that performed a continuous statistical learning task in which respondents were asked to predict the next event in a sequence of musical chords. An underlying transitional probability matrix governed the chord sequences. Each participant performed both a positive- and negative-valence block of this task, where blocks differed in the precise musical chords used. A pilot experiment established that the sequences from both blocks evoked their intended perceived valence. Furthermore, cognitive assessment (Raven’s advanced matrices) as well as mood scores (DASS-21) were collected. Bayesian mixed effects models revealed that participants were able to extract the underlying transitional probabilities and that higher cognitive ability predicted higher performance. Furthermore, there was strong evidence that the depression, anxiety, and stress subscales all predicted learning trajectories, and interacted with stimulus valence. Thus, the present results show that information processing differences in a musical context are consistent with the phenomenology of mood disorders. The present study is one step towards a non-invasive musical tool to screen for mood disorders

Structured decision making in adult protective services

The focus of this research project was to evaluate individual stakeholder perceptions of the implementation of Structured Decision Making (SDM) as a tool in risk assessment of elder abuse/neglect case referrals in Riverside County Adult Protective Services (APS). The researchers interviewed ten social workers from Riverside County APS, from line workers up to Deputy Director. Constructivist theory was used to develop a joint construct which indicated that the primary benefit of SDM was consistency. Consistency resulted in safety and proctection of the client, social worker and the agency

Prefrontal High Gamma in ECoG Tags Periodicity of Musical Rhythms in Perception and Imagination

Rhythmic auditory stimuli are known to elicit matching activity patterns in neural populations. Furthermore, recent research has established the particular importance of high-gamma brain activity in auditory processing by showing its involvement in auditory phrase segmentation and envelope tracking. Here, we use electrocorticographic (ECoG) recordings from eight human listeners to see whether periodicities in high-gamma activity track the periodicities in the envelope of musical rhythms during rhythm perception and imagination. Rhythm imagination was elicited by instructing participants to imagine the rhythm to continue during pauses of several repetitions. To identify electrodes whose periodicities in high-gamma activity track the periodicities in the musical rhythms, we compute the correlation between the autocorrelations (ACCs) of both the musical rhythms and the neural signals. A condition in which participants listened to white noise was used to establish a baseline. High-gamma autocorrelations in auditory areas in the superior temporal gyrus and in frontal areas on both hemispheres significantly matched the autocorrelations of the musical rhythms. Overall, numerous significant electrodes are observed on the right hemisphere. Of particular interest is a large cluster of electrodes in the right prefrontal cortex that is active during both rhythm perception and imagination. This indicates conscious processing of the rhythms\u27 structure as opposed to mere auditory phenomena. The autocorrelation approach clearly highlights that high-gamma activity measured from cortical electrodes tracks both attended and imagined rhythms

Editorial: Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures

International audienc

Duration of disease does not equally influence all aspects of quality of life in Parkinson’s disease

Health related quality of life (HRQoL) is negatively impacted in patients suffering from Parkinsons disease (PD). For the specific components that comprise HRQoL, the relationship between clinical variables, such as disease duration, is not fully characterized. In this cross-sectional study (n=302), self-reported HRQoL on the Parkinsons Disease Questionnaire (PDQ-39) was evaluated as a global construct as well as individual subscale scores. HRQoL was compared in three groups: those within 5years of diagnosis, those within 6-10years of diagnosis, and those greater than 11years since diagnosis. Non-parametric analyses revealed lower HRQoL with increasing disease duration when assessed as a global construct. However, when subscales were evaluated, difficulties with bodily discomfort and cognitive complaints were comparable in individuals in the 1-5years and 6-10year duration groups. Exploratory regression analyses suggested disease duration does explain unique variance in some subscales, even after controlling for Hoehn and Yahr stage and neuropsychiatric features. Our findings show that HRQoL domains in PD patients are affected differentially across the duration of the disease. Clinicians and researchers may need to tailor interventions intended to improve HRQoL at different domains as the disease progresses