Cortical tracking of surprisal during continuous speech comprehension

Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as a of the precision of such a prediction. However, they have focussed on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension a listener hears many successive words whose predictability and precision vary over a large range. Here we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech, and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network, and through relating these speech features to electroencephalographic (EEG) responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies including the delta band as well as in the higher-frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension

Mind over chatter: plastic up-regulation of the fMRI alertness network by EEG neurofeedback

EEG neurofeedback (NFB) is a brain-computer interface (BCI) approach used to shape brain oscillations by means of real-time feedback from the electroencephalogram (EEG), which is known to reflect neural activity across cortical networks. Although NFB is being evaluated as a novel tool for treating brain disorders, evidence is scarce on the mechanism of its impact on brain function. In this study with 34 healthy participants, we examined whether, during the performance of an attentional auditory oddball task, the functional connectivity strength of distinct fMRI networks would be plastically altered after a 30-min NFB session of alpha-band reduction (n=17) versus a sham-feedback condition (n=17). Our results reveal that compared to sham, NFB induced a specific increase of functional connectivity within the alertness/salience network (dorsal anterior and mid cingulate), which was detectable 30 minutes after termination of training. Crucially, these effects were significantly correlated with reduced mind-wandering 'on-task' and were coupled to NFB-mediated resting state reductions in the alpha-band (8-12 Hz). No such relationships were evident for the sham condition. Although group default-mode network (DMN) connectivity was not significantly altered following NFB, we observed a positive association between modulations of resting alpha amplitude and precuneal connectivity, both correlating positively with frequency of mind-wandering. Our findings demonstrate a temporally direct, plastic impact of NFB on large-scale brain functional networks, and provide promising neurobehavioral evidence supporting its use as a noninvasive tool to modulate brain function in health and disease

Brain areas associated with visual spatial attention display topographic organization during auditory spatial attention

Spatially selective modulation of alpha power (8–14 Hz) is a robust finding in electrophysiological studies of visual attention, and has been recently generalized to auditory spatial attention. This modulation pattern is interpreted as reflecting a top-down mechanism for suppressing distracting input from unattended directions of sound origin. The present study on auditory spatial attention extends this interpretation by demonstrating that alpha power modulation is closely linked to oculomotor action. We designed an auditory paradigm in which participants were required to attend to upcoming sounds from one of 24 loudspeakers arranged in a circular array around the head. Maintaining the location of an auditory cue was associated with a topographically modulated distribution of posterior alpha power resembling the findings known from visual attention. Multivariate analyses allowed the prediction of the sound location in the horizontal plane. Importantly, this prediction was also possible, when derived from signals capturing saccadic activity. A control experiment on auditory spatial attention confirmed that, in absence of any visual/auditory input, lateralization of alpha power is linked to the lateralized direction of gaze. Attending to an auditory target engages oculomotor and visual cortical areas in a topographic manner akin to the retinotopic organization associated with visual attention

Lower Beta: A Central Coordinator of Temporal Prediction in Multimodal Speech

How the brain decomposes and integrates information in multimodal speech perception is linked to oscillatory dynamics. However, how speech takes advantage of redundancy between different sensory modalities, and how this translates into specific oscillatory patterns remains unclear. We address the role of lower beta activity (~20 Hz), generally associated with motor functions, as an amodal central coordinator that receives bottom-up delta-theta copies from specific sensory areas and generate top-down temporal predictions for auditory entrainment. Dissociating temporal prediction from entrainment may explain how and why visual input benefits speech processing rather than adding cognitive load in multimodal speech perception. On the one hand, body movements convey prosodic and syllabic features at delta and theta rates (i.e., 1–3 Hz and 4–7 Hz). On the other hand, the natural precedence of visual input before auditory onsets may prepare the brain to anticipate and facilitate the integration of auditory delta-theta copies of the prosodic-syllabic structure. Here, we identify three fundamental criteria based on recent evidence and hypotheses, which support the notion that lower motor beta frequency may play a central and generic role in temporal prediction during speech perception. First, beta activity must respond to rhythmic stimulation across modalities. Second, beta power must respond to biological motion and speech-related movements conveying temporal information in multimodal speech processing. Third, temporal prediction may recruit a communication loop between motor and primary auditory cortices (PACs) via delta-to-beta cross-frequency coupling. We discuss evidence related to each criterion and extend these concepts to a beta-motivated framework of multimodal speech processing

The Impact of Hand Movement Velocity on Cognitive Conflict Processing in a 3D Object Selection Task in Virtual Reality.

Detecting and correcting incorrect body movements is an essential part of everyday interaction with one's environment. The human brain provides a monitoring system that constantly controls and adjusts our actions according to our surroundings. However, when our brain's predictions about a planned action do not match the sensory inputs resulting from that action, cognitive conflict occurs. Much is known about cognitive conflict in 1D/2D environments; however, less is known about the role of movement characteristics associated with cognitive conflict in 3D environment. Hence, we devised an object selection task in a virtual reality (VR) environment to test how the velocity of hand movements impacts human brain responses. From a series of analyses of EEG recordings synchronized with motion capture, we found that the velocity of the participants' hand movements modulated the brain's response to proprioceptive feedback during the task and induced a prediction error negativity (PEN). Additionally, the PEN originates in the anterior cingulate cortex and is itself modulated by the ballistic phase of the hand's movement. These findings suggest that velocity is an essential component of integrating hand movements with visual and proprioceptive information during interactions with real and virtual objects

Oszillatorische Gamma-Band-Aktivität bei der Verarbeitung auditorischer Reize im Kurzzeitgedächtnis im MEG

Recent studies have suggested an important role of cortical gamma oscillatory activity (30-100 Hz) as a correlate of encoding, maintaining and retrieving auditory, visual or tactile information in and from memory. It was shown that these cortical stimulus representations were modulated by attention processes. Gamma-band activity (GBA) occurred as an induced response peaking at approximately 200-300 ms after stimulus presentation. Induced cortical responses appear as non-phase-locked activity and are assumed to reflect active cortical processing rather than passive perception. Induced GBA peaking 200-300 ms after stimulus presentation has been assumed to reflect differences between experimental conditions containing various stimuli. By contrast, the relationship between specific oscillatory signals and the representation of individual stimuli has remained unclear. The present study aimed at the identification of such stimulus-specific gamma-band components. We used magnetoencephalography (MEG) to assess gamma activity during an auditory spatial delayed matching-to-sample task. 28 healthy adults were assigned to one of two groups R and L who were presented with only right- or left-lateralized sounds, respectively. Two sample stimuli S1 with lateralization angles of either 15° or 45° deviation from the midsagittal plane were used in each group. Participants had to memorize the lateralization angle of S1 and compare it to a second lateralized sound S2 presented after an 800-ms delay phase. S2 either had the same or a different lateralization angle as S1. After the presentation of S2, subjects had to indicate whether S1 and S2 matched or not. Statistical probability mapping was applied to the signals at sensor level to identify spectral amplitude differences between 15° and 45° stimuli. We found distinct gamma-band components reflecting each sample stimulus with center frequencies ranging between 59 and 72 Hz in different sensors over parieto-occipital cortex contralateral to the side of stimulation. These oscillations showed maximal spectral amplitudes during the middle 200-300 ms of the delay phase and decreased again towards its end. Additionally, we investigated correlations between the activation strength of the gamma-band components and memory task performance. The magnitude of differentiation between oscillatory components representing 'preferred' and 'nonpreferred' stimuli during the final 100 ms of the delay phase correlated positively with task performance. These findings suggest that the observed gamma-band components reflect the activity of neuronal networks tuned to specific auditory spatial stimulus features. The activation of these networks seems to contribute to the maintenance of task-relevant information in short-term memory.Ergebnisse aus aktuellen Studien legen nahe, dass kortikale oszillatorische Aktivität im Gamma-Bereich (30-100 Hz) eine wichtige Rolle für verschiedene kognitive Prozesse spielt. Dazu zählen das Kodieren, die Aufrechterhaltung und der Abruf auditorischer, visueller oder taktiler Informationen in das bzw. aus dem Gedächtnis. Es konnte gezeigt werden, dass diese kortikale Aktivität durch Aufmerksamkeitsprozesse beeinflusst wird. Gamma-Aktivität trat bei vorangegangenen Untersuchungen als induzierte Antwort ca. 200-300 ms nach Stimuluspräsentation auf. Es wird angenommen, dass diese nicht phasengebundenen kortikalen Reizantworten aktive kortikale Verarbeitungs-prozesse widerspiegeln. In früheren Studien wurde induzierte Gamma-Aktivität während der Aufrechterhaltung von Stimulusinformationen über Regionen gefunden, die an der Verarbeitung aufgabenrelevanter Reizmerkmale beteiligt sind. Diese Antworten im Gamma-Bereich spiegelten Unterschiede zwischen verschieden experimentellen Bedingungen wider, jedoch ist wenig über die Repräsentation spezifischer Stimuluseigenschaften durch Gamma-Aktivität bekannt. Mit der vorliegenden Studie haben wir versucht, solche stimulus spezifischen Gamma-Komponenten zu untersuchen. Dafür verwendeten wir Magnetenzephalographie (MEG) und eine auditorische räumliche “delayed matching-to-sample“ Aufgabe. 28 gesunde Erwachsene wurden dabei zwei verschiedenen Gruppen zugeordnet. Gruppe R bekam rechtslateralisierte Stimuli präsentiert, während diese in Gruppe L linkslateralisiert waren. Dabei unterschieden sich die Reize nur in ihrer räumlichen Charakteristik, die Klangmuster blieben unverändert. In beiden Gruppen wurden zwei Beispielstimuli S1 mit Lateralisierungswinkeln von 15° bzw. 45° verwendet. Die Probanden mussten sich den Lateralisierungswinkel von S1 merken und anschließend mit einem zweiten Stimulus S2, der nach einer Verzögerungsphase von 800 ms präsentiert wurde, vergleichen. S2 hatte dabei entweder den gleichen Lateralisierungswinkel wie S1, oder unterschied sich darin von dem ersten Stimulus. Nach der Präsentation von S2 mussten die Probanden signalisieren, ob die Lateralisierungswinkel der beiden Stimuli übereinstimmten oder nicht. Die Signale der einzelnen Sensoren wurden mit einem statistischen Wahrscheinlichkeitsmapping untersucht. Dabei wollten wir Unterschiede in der spektralen Amplitude für Stimuli mit 15° bzw. 45° Lateralisierungswinkel identifizieren. Wir konnten spezifische Gamma-Aktivität für alle Beispielstimuli nachweisen. Die Signale wurden im Bereich von 59-72 Hz gefunden und waren über dem parieto-okzipitalen Kortex jeweils kontralateral zur stimulierten Seite lokalisiert. Die maximalen Spektralamplituden dieser Oszillationen traten während der mittleren 200-300 ms der Verzögerungsphase auf und nahmen zu ihrem Ende hin ab. Zusätzlich haben wir Korrelationen zwischen der Aktivierungsstärke der Gamma-Komponenten und dem Abschneiden bei der Gedächtnisaufgabe untersucht. Dabei zeigte sich, dass der Unterschied der oszillatorischen Antworten auf bevorzugte und nicht-bevorzugte Stimuli während der letzten 100 ms der Verzögerungsphase positiv mit der Leistung in der Gedächtnisaufgabe korrelierte. Diese Ergebnisse sprechen dafür, dass die beobachteten Gamma Komponenten die Aktivität neuronaler Netzwerke, die auf die Verarbeitung räumlicher auditorischer Information spezialisiert sind, widerspiegeln. Die Aktivierung dieser Netzwerke scheint zur Aufrechterhaltung aufgabenbezogener Information im Kurzzeitgedächtnis beizutragen

The directional effect of target position on spatial selective auditory attention

Spatial selective auditory attention plays a crucial role in listening in a mixture of competing speech sounds. Previous neuroimaging studies have reported alpha band neural activity modulated by auditory attention, along with the alpha lateralization corresponding to attentional focus. A greater cortical representation of the attended speech envelope compared to the ignored speech envelope was also found, a phenomenon known as \u27neural speech tracking’. However, little is known about the neural activities when attentional focus is directed on speech sounds from behind the listener, even though understanding speech from behind is a common and essential aspect of daily life. The objectives of this study are to investigate the impact of four distinct target positions (left, right, front, and particularly, behind) on spatial selective auditory attention by concurrently assessing 1) spatial selective speech identification, 2) oscillatory alpha-band power, and 3) neural speech tracking. Fifteen young adults with normal hearing (NH) were enrolled in this study (M = 21.40, ages 18-29; 10 females). The selective speech identification task indicated that the target position presented at back was the most challenging condition, followed by the front condition, with the lateral condition being the least demanding. The normalized alpha power was modulated by target position and the power was significantly lateralized to either the left or right side, not the front and back. The parieto-occipital alpha power in front-back configuration was significantly lower than the results for left-right listening configuration and the normalized alpha power in the back condition was significantly higher than in the front condition. The speech tracking function of to-be-attended speech envelope was affected by the direction of ix target stream. The behavioral outcome (selective speech identification) was correlated with parieto-occipital alpha power and neural speech tracking correlation coefficient as neural correlates of auditory attention, but there was no significant correlation between alpha power and neural speech tracking. The results suggest that in addition to existing mechanism theories, it might be necessary to consider how our brain responds depending on the location of the sound in order to interpret the neural correlates and behavioral consequences in a meaningful way as well as a potential application of neural speech tracking in studies on spatial selective hearing

Mindfulness Modulated Attention with Neurofeedback in Multiple Object Tracking

Attention determines what we selectively perceive out of all available stimuli. The multiple-object tracking paradigm is a way of examining divided attention for object tracking in a complex visual scene. Mindfulness is a broad term for a set of diverse and specific methods for distinct attentional engagement and is one technique for increasing visual attentional ability and decreasing distractibility. Neurofeedback can be a way of enhancing mindfulness training for novice participants. This study examined the relationship between attention and mindfulness with neurofeedback through performance on a multiple-object tracking task and the Five Facet Mindfulness Questionnaire. We examined the effectiveness of using a brief mindfulness session to bring about state mindfulness and cognitive enhancement. All participants (N=90) performed a session of the multiple-object tracking task before and after either a mindfulness or relaxation intervention. Additionally half of the participants in the mindfulness training condition received neurofeedback. Results demonstrated that a single, brief mindfulness training session with neurofeedback was successful in increasing divided attention ability and was sufficient for bringing about an increased mindfulness state. An effect of mindfulness without neurofeedback on attention was not found. Results have implications for the use of brief mindfulness practices in a laboratory setting that could be applicable to a real world setting and the feasibility of neurofeedback as a mindfulness training tool

Assessing the transfer of video game play versus attention training using 3D-Multiple Object Tracking

Durant la dernière décennie, la recherche sur les jeux vidéo et leur implication sur les habiletés perceptivo-cognitives a gagné en intérêt. Plusieurs études ont démontré que les jeux vidéo (particulièrement les jeux d’action) possèdent la capacité d’influencer et d’améliorer différentes aptitudes perceptives et cognitives telles que l’attention visuo-spatiale, la vitesse de traitement de l’information, la mémoire visuelle à court terme ainsi que la poursuite d’objets en mouvement. Cependant, plusieurs autres études n’ont pas réussi à reproduire les mêmes résultats. D’un autre côté, un nouveau type d’entraînement perceptivo-cognitif, nommé 3-Dimensional Multiple-Object Tracking (3D-MOT), et qui consiste à traiter des scènes visuelles dynamiques dénuées de contexte, a démontré son implication sur différents types d’attention, la mémoire de travail ainsi que la vitesse de traitement de l’information. L’étude actuelle a examiné quatre groupes de joueurs inexpérimentés qui s’entrainaient durant 10 séances à l’aide d’un exercice perceptivo-cognitif (3D-MOT), ou d’un jeu de haut niveau visuel (jeu vidéo d’action : Call of Duty), de bas niveau visuel (Tetris) ou d’un jeu non-visuel (Sudoku). Des mesures d’électroencéphalographie quantitative et des tests neuropsychologiques effectués avant et après l’entraînement ont démontré que le 3D-MOT, par comparaison aux autres jeux testés, améliorait de façon plus efficace les fonctions reliées à l’attention, la mémoire de travail ainsi que la vitesse de traitement de l’information. Pour la première fois, cette étude démontre que l’entraînement non-contextuel de 3D-MOT améliore les habiletés perceptivo-cognitives plus efficacement que l’entraînement à des jeux de divertissement tels que les jeux vidéo.In the past decade, research on video games and their implications on cognitive abilities have gained significant interest. Various studies suggest that video games (in particular action video games) have the inherent ability to influence and improve attentional abilities such as visual spatial attention, processing speed, visual short-term memory and multiple-object tracking. However, many other studies have been unable to replicate similar results. On the other hand, a recent cognitive enhancement tool that is visually dynamic and void of context called 3-Dimensional Multiple-Object tracking (3D-MOT), has demonstrated robust effects on cognitive-perceptual abilities such as divided, selective, and sustained attention as well as working memory and information processing speed. The current study examines four groups of non-video game players that train for 10 sessions on the cognitive enhancing technique (3D-MOT) or on one of three different visually stimulating games: highly visually stimulating game (Call of Duty), lowly visually stimulating game (Tetris), or non-visually stimulating puzzle (Sudoku). A battery of cognitive tests and quantitative electroencephalography preformed before and after training, demonstrated that training on 3D-MOT improved cognitive functions related to attention, working memory, and visual information processing compared to video games. For the first time, this study demonstrated that non-contextual training with 3D-MOT improves perceptual-cognitive abilities more efficiently than video game playing