Decoding Electrophysiological Correlates of Selective Attention by Means of Circular Data

Sustaining our attention to a relevant sensory input in a complex listening environment, is of great importance for a successful auditory communication. To avoid the overload of the auditory system, the importance of the stimuli is estimated in the higher levels of the auditory system. Based on these information, the attention is drifted away from the irrelevant and unimportant stimuli. Long-term habituation, a gradual process independent from sensory adaptation, plays a major role in drifting away our attention from irrelevant stimuli. A better understanding of attention-modulated neural activity is important for shedding light on the encoding process of auditory streams. For instance, these information can have a direct impact on developing smarter hearing aid devices in which more accurate objective measures can be used to re ect the hearing capabilities of patients with hearing pathologies. As an example, an objective measures of long-term habituation with respect to di erent level of sound stimuli can be used more accurately for adjustment of hearing aid devices in comparison to verbal reports. The main goal of this thesis is to analyze the neural decoding signatures of long-term habituation and neural modulations of selective attention by exploiting circular regularities in electrophysiological (EEG) data, in which we can objectively measure the level of attentional-binding to di erent stimuli. We study, in particular, the modulations of the instantaneous phase (IP) in event related potentials (ERPs) over trials for di erent experimental settings. This is in contrast to the common approach where the ERP component of interest is computed through averaging a su ciently large number of ERP trials. It is hypothesized that a high attentional binding to a stimulus is related to a high level of IP cluster. As the attention binding reduces, IP is spread more uniformly on a unit circle. This work is divided into three main parts. In the initial part, we investigate the dynamics of long-term habituation with di erent acoustical stimuli (soft vs. loud) over ERP trials. The underlying temporal dynamics in IP and the level of phase cluster of the ERPs are assessed by tting circular probability functions (pdf) over data segments. To increase the temporal resolution of detecting times at which a signi cant change in IP occurs, an abrupt change point model at di erent pure-tone stimulations is used. In a second study, we improve upon the results and methodology by relaxing some of the constrains in order to integrate the gradual process of long-term habituation into the model. For this means, a Bayesian state-space model is proposed. In all of the aforementioned studies, we successfully classi ed between di erent stimulation levels, using solely the IP of ERPs over trials. In the second part of the thesis, the experimental setting is expanded to contain longer and more complex auditory stimuli as in real-world scenarios. Thereby, we study the neural-correlates of attention in spontaneous modulations of EEG (ongoing activity) which uses the complete temporal resolution of the signal. We show a mapping between the ERP results and the ongoing EEG activity based on IP. A Markov-based model is developed for removing spurious variations that can occur in ongoing signals. We believe the proposed method can be incorporated as an important preprocessing step for a more reliable estimation of objective measures of the level of selective attention. The proposed model is used to pre-process and classify between attending and un-attending states in a seminal dichotic tone detection experiment. In the last part of this thesis, we investigate the possibility of measuring a mapping between the neural activities of the cortical laminae with the auditory evoked potentials (AEP) in vitro. We show a strong correlation between the IP of AEPs and the neural activities at the granular layer, using mutual information.Die Aufmerksamkeit auf ein relevantes auditorisches Signal in einer komplexen H orumgebung zu lenken ist von gro er Bedeutung f ur eine erfolgreiche akustische Kommunikation. Um eine Uberlastung des H orsystems zu vermeiden, wird die Bedeutung der Reize in den h oheren Ebenen des auditorischen Systems bewertet. Basierend auf diesen Informationen wird die Aufmerksamkeit von den irrelevanten und unwichtigen Reizen abgelenkt. Dabei spielt die sog. Langzeit- Habituation, die einen graduellen Prozess darstellt der unabh angig von der sensorischen Adaptierung ist, eine wichtige Rolle. Ein besseres Verst andnis der aufmerksamkeits-modulierten neuronalen Aktivit at ist wichtig, um den Kodierungsprozess von sog. auditory streams zu beleuchten. Zum Beispiel k onnen diese Informationen einen direkten Ein uss auf die Entwicklung intelligenter H orsysteme haben bei denen genauere, objektive Messungen verwendet werden k onnen, um die H orf ahigkeiten von Patienten mit H orpathologien widerzuspiegeln. So kann beispielsweise ein objektives Ma f ur die Langzeit- Habituation an unterschiedliche Schallreize genutzt werden um - im Vergleich zu subjektiven Selbsteinsch atzungen - eine genauere Anpassung der H orsysteme zu erreichen. Das Hauptziel dieser Dissertation ist die Analyse neuronaler Dekodierungssignaturen der Langzeit- Habituation und neuronaler Modulationen der selektiver Aufmerksamkeit durch Nutzung zirkul arer Regularit aten in elektroenzephalogra schen Daten, in denen wir objektiv den Grad der Aufmerksamkeitsbindung an verschiedene Reize messen k onnen. Wir untersuchen insbesondere die Modulation der Momentanphase (engl. Instantaneous phase, IP) in ereigniskorrelierten Potenzialen (EKPs) in verschiedenen experimentellen Settings. Dies steht im Gegensatz zu dem traditionellen Ansatz, bei dem die interessierenden EKP-Komponenten durch Mittelung einer ausreichend gro en Anzahl von Einzelantworten im Zeitbereich ermittelt werden. Es wird vermutet, dass eine hohe Aufmerksamkeitsbindung an einen Stimulus mit einem hohen Grad an IP-Clustern verbunden ist. Nimmt die Aufmerksamkeitsbindung hingegen ab, so ist die Momentanphase uniform auf dem Einheitskreis verteilt. Diese Arbeit gliedert sich in drei Teile. Im ersten Teil untersuchen wir die Dynamik der Langzeit-Habituation mit verschiedenen akustischen Reizen (leise vs. laut) in EKP-Studien. Die zugrundeliegende zeitliche Dynamik der Momentanphase und die Ebene des Phasenclusters der EKPs werden durch die Anpassung von zirkul aren Wahrscheinlichkeitsfunktionen (engl. probability density function, pdf) uber Datensegmente bewertet. Mithilfe eines sog. abrupt change-point Modells wurde die zeitliche Au osung der Daten erh oht, sodass signi kante Anderungen in der Momentanphase bei verschiedenen Reintonstimulationen detektierbar sind. In einer zweiten Studie verbessern wir die Ergebnisse und die Methodik, indem wir einige der Einschr ankungen lockern, um den gradualen Prozess der Langzeit-Habituation in das abrupt changepoint Modell zu integrieren. Dazu wird ein bayes`sches Zustands-Raum-Modell vorgeschlagen. In den zuvor genannten Studien konnte erfolgreich mithilfe der Momentanphase zwischen verschiedenen Stimulationspegeln unterschieden werden. Im zweiten Teil der Arbeit wird der experimentelle Rahmen erweitert, um komplexere auditorische Reize wie in realen H orsituationen untersuchen zu k onnen. Dabei analysieren wir die neuronalen Korrelate der Aufmerksamkeit anhand spontaner Modulationen der kontinuierlichen EEG-Aktivit at, die eine zeitliche Au osung erm oglicht. Wir zeigen eine Abbildung zwischen den EKP-Ergebnissen und der kontinuierlichen EEG-Aktivit at auf Basis der Momentanphase. Ein Markov-basiertes Modell wird entwickelt, um st orende Variationen zu entfernen, die in kontinuierlichen EEG-Signalen auftreten k onnen. Wir glauben, dass die vorgeschlagene Methode als wichtiger Vorverarbeitungsschritt zur soliden objektiven Absch atzung des Aufmerksamkeitsgrades mithilfe von EEG-Daten verwendet werden kann. In einem dichotischen Tonerkennungsexperiment wird das vorgeschlagene Modell zur Vorverarbeitung der EEG-Daten und zur Klassi zierung zwischen gerichteten und ungerichteten Aufmerksamkeitszust anden erfolgreich verwendet. Im letzten Teil dieser Arbeit untersuchen wir den Zusammenhang zwischen den neuronalen Aktivit aten der kortikalen Laminae und auditorisch evozierten Potentialen (AEP) in vitro im Tiermodell. Wir zeigen eine starke Korrelation zwischen der Momentanphase der AEPs und den neuronalen Aktivit aten in der Granularschicht unter Verwendung der Transinformation

True zero-training brain-computer interfacing: an online study

Despite several approaches to realize subject-to-subject transfer of pre-trained classifiers, the full performance of a Brain-Computer Interface (BCI) for a novel user can only be reached by presenting the BCI system with data from the novel user. In typical state-of-the-art BCI systems with a supervised classifier, the labeled data is collected during a calibration recording, in which the user is asked to perform a specific task. Based on the known labels of this recording, the BCI's classifier can learn to decode the individual's brain signals. Unfortunately, this calibration recording consumes valuable time. Furthermore, it is unproductive with respect to the final BCI application, e.g. text entry. Therefore, the calibration period must be reduced to a minimum, which is especially important for patients with a limited concentration ability. The main contribution of this manuscript is an online study on unsupervised learning in an auditory event-related potential (ERP) paradigm. Our results demonstrate that the calibration recording can be bypassed by utilizing an unsupervised trained classifier, that is initialized randomly and updated during usage. Initially, the unsupervised classifier tends to make decoding mistakes, as the classifier might not have seen enough data to build a reliable model. Using a constant re-analysis of the previously spelled symbols, these initially misspelled symbols can be rectified posthoc when the classifier has learned to decode the signals. We compare the spelling performance of our unsupervised approach and of the unsupervised posthoc approach to the standard supervised calibration-based dogma for n = 10 healthy users. To assess the learning behavior of our approach, it is unsupervised trained from scratch three times per user. Even with the relatively low SNR of an auditory ERP paradigm, the results show that after a limited number of trials (30 trials), the unsupervised approach performs comparably to a classic supervised model

Moregrasp: Restoration of Upper Limb Function in Individuals with High Spinal Cord Injury by Multimodal Neuroprostheses for Interaction in Daily Activities

The aim of the MoreGrasp project is to develop a noninvasive, multimodal user interface including a brain-computer interface (BCI) for intuitive control of a grasp neuroprosthesis to support individuals with high spinal cord injury (SCI) in everyday activities. We describe the current state of the project, including the EEG system, preliminary results of natural movements decoding in people with SCI, the new electrode concept for the grasp neuroprosthesis, the shared control architecture behind the system and the implementation of a user-centered design

Simple acoustic features can explain phoneme-based predictions of cortical responses to speech

When we listen to speech, we have to make sense of a waveform of sound pressure. Hierarchical models of speech perception assume that, to extract semantic meaning, the signal is transformed into unknown, intermediate neuronal representations. Traditionally, studies of such intermediate representations are guided by linguistically defined concepts, such as phonemes. Here, we argue that in order to arrive at an unbiased understanding of the neuronal responses to speech, we should focus instead on representations obtained directly from the stimulus. We illustrate our view with a data-driven, information theoretic analysis of a dataset of 24 young, healthy humans who listened to a 1 h narrative while their magnetoencephalogram (MEG) was recorded. We find that two recent results, the improved performance of an encoding model in which annotated linguistic and acoustic features were combined and the decoding of phoneme subgroups from phoneme-locked responses, can be explained by an encoding model that is based entirely on acoustic features. These acoustic features capitalize on acoustic edges and outperform Gabor-filtered spectrograms, which can explicitly describe the spectrotemporal characteristics of individual phonemes. By replicating our results in publicly available electroencephalography (EEG) data, we conclude that models of brain responses based on linguistic features can serve as excellent benchmarks. However, we believe that in order to further our understanding of human cortical responses to speech, we should also explore low-level and parsimonious explanations for apparent high-level phenomena

Oscillatory Mechanisms of Preparing for Visual Distraction

Evidence shows that observers preactivate a target representation in preparation of a visual selection task. In this study, we addressed the question if and how preparing to ignore an anticipated distractor differs from preparing for an anticipated target. We measured EEG while participants memorized a laterally presented color, which was cued to be either a target or a distractor in two subsequent visual search tasks. Decoding the location of items in the search display from EOG channels revealed that, initially, the anticipated distractor attracted attention and could only be ignored later during the trial. This suggests that distractors could not be suppressed in advance but were represented in an active, attention-guiding format. Consistent with this, lateralized posterior alpha power did not dissociate between target and distractor templates during the delay periods, suggesting similar encoding and maintenance. However, distractor preparation did lead to relatively enhanced nonlateralized posterior alpha power, which appeared to gate sensory processing at search display onset to prevent attentional capture in general. Finally, anticipating distractors also led to enhanced midfrontal theta power during the delay period, a signal that was predictive of how strongly both target and distractor were represented in the search display. Together, our results speak against a distractor-specific advance inhibitory template, thus contrary to the preactivation of specific target templates. Rather, we demonstrate a general selection suppression mechanism, which serves to prevent initial involuntary capture by anticipated distracting input

Efficient Solutions to High-Dimensional and Nonlinear Neural Inverse Problems

Development of various data acquisition techniques has enabled researchers to study the brain as a complex system and gain insight into the high-level functions performed by different regions of the brain. These data are typically high-dimensional as they pertain to hundreds of sensors and span hours of recording. In many experiments involving sensory or cognitive tasks, the underlying cortical activity admits sparse and structured representations in the temporal, spatial, or spectral domains, or combinations thereof. However, current neural data analysis approaches do not take account of sparsity in order to harness the high-dimensionality. Also, many existing approaches suffer from high bias due to the heavy usage of linear models and estimation techniques, given that cortical activity is known to exhibit various degrees of non-linearity. Finally, the majority of current methods in computational neuroscience are tailored for static estimation in batch-mode and offline settings, and with the advancement of brain-computer interface technologies, these methods need to be extended to capture neural dynamics in a real-time fashion. The objective of this dissertation is to devise novel algorithms for real-time estimation settings and to incorporate the sparsity and non-linear properties of brain activity for providing efficient solutions to neural inverse problems involving high-dimensional data. Along the same line, our goal is to provide efficient representations of these high-dimensional data that are easy to interpret and assess statistically. First, we consider the problem of spectral estimation from binary neuronal spiking data. Due to the non-linearities involved in spiking dynamics, classical spectral representation methods fail to capture the spectral properties of these data. To address this challenge, we integrate point process theory, sparse estimation, and non-linear signal processing methods to propose a spectral representation modeling and estimation framework for spiking data. Our model takes into account the sparse spectral structure of spiking data, which is crucial in the analysis of electrophysiology data in conditions such as sleep and anesthesia. We validate the performance of our spectral estimation framework using simulated spiking data as well as multi-unit spike recordings from human subjects under general anesthesia. Next, we tackle the problem of real-time auditory attention decoding from electroencephalography (EEG) or magnetoencephalography (MEG) data in a competing-speaker environment. Most existing algorithms for this purpose operate offline and require access to multiple trials for a reliable performance; hence, they are not suitable for real-time applications. To address these shortcomings, we integrate techniques from state-space modeling, Bayesian filtering, and sparse estimation to propose a real-time algorithm for attention decoding that provides robust, statistically interpretable, and dynamic measures of the attentional state of the listener. We validate the performance of our proposed algorithm using simulated and experimentally-recorded M/EEG data. Our analysis reveals that our algorithms perform comparable to the state-of-the-art offline attention decoding techniques, while providing significant computational savings. Finally, we study the problem of dynamic estimation of Temporal Response Functions (TRFs) for analyzing neural response to auditory stimuli. A TRF can be viewed as the impulse response of the brain in a linear stimulus-response model. Over the past few years, TRF analysis has provided researchers with great insight into auditory processing, specially under competing speaker environments. However, most existing results correspond to static TRF estimates and do not examine TRF dynamics, especially in multi-speaker environments with attentional modulation. Using state-space models, we provide a framework for a robust and comprehensive dynamic analysis of TRFs using single trial data. TRF components at specific lags may exhibit peaks which arise, persist, and disappear over time according to the attentional state of the listener. To account for this specific behavior in our model, we consider a state-space model with a Gaussian mixture process noise, and devise an algorithm to efficiently estimate the process noise parameters from the recorded M/EEG data. Application to simulated and recorded MEG data shows that the {proposed state-space modeling and inference framework can reliably capture the dynamic changes in the TRF, which can in turn improve our access to the attentional state in competing-speaker environments

Predictive decoding of neural data

In the last five decades the number of techniques available for non-invasive functional imaging has increased dramatically. Researchers today can choose from a variety of imaging modalities that include EEG, MEG, PET, SPECT, MRI, and fMRI. This doctoral dissertation offers a methodology for the reliable analysis of neural data at different levels of investigation. By using statistical learning algorithms the proposed approach allows single-trial analysis of various neural data by decoding them into variables of interest. Unbiased testing of the decoder on new samples of the data provides a generalization assessment of decoding performance reliability. Through consecutive analysis of the constructed decoder\u27s sensitivity it is possible to identify neural signal components relevant to the task of interest. The proposed methodology accounts for covariance and causality structures present in the signal. This feature makes it more powerful than conventional univariate methods which currently dominate the neuroscience field. Chapter 2 describes the generic approach toward the analysis of neural data using statistical learning algorithms. Chapter 3 presents an analysis of results from four neural data modalities: extracellular recordings, EEG, MEG, and fMRI. These examples demonstrate the ability of the approach to reveal neural data components which cannot be uncovered with conventional methods. A further extension of the methodology, Chapter 4 is used to analyze data from multiple neural data modalities: EEG and fMRI. The reliable mapping of data from one modality into the other provides a better understanding of the underlying neural processes. By allowing the spatial-temporal exploration of neural signals under loose modeling assumptions, it removes potential bias in the analysis of neural data due to otherwise possible forward model misspecification. The proposed methodology has been formalized into a free and open source Python framework for statistical learning based data analysis. This framework, PyMVPA, is described in Chapter 5