The neural signature of information regularity in temporally extended event sequences

Statistical regularities exist at different timescales in temporally unfolding event sequences. Recent studies have identified brain regions that are sensitive to the levels of regularity in sensory inputs, enabling the brain to construct a representation of environmental structure and adaptively generate actions or predictions. However, the temporal specificity of the statistical regularity to which the brain responds remains largely unknown. This uncertainty applies to the regularities of sensory inputs as well as instrumental actions. Here, we used fMRI to investigate the neural correlates of regularity in sequences of task events and action selections in a visuomotor choice task. We quantified timescale-dependent regularity measures by calculating Shannon's entropy and surprise from a sliding-window of consecutive task events and actions. Activity in the frontopolar cortex negatively correlated with the entropy in action selection, while activity in the temporoparietal junction, the striatum, and the cerebellum negatively correlated with the entropy in stimulus events at longer timescales. In contrast, activity in the supplementary motor area, the superior frontal gyrus, and the superior parietal lobule was positively correlated with the surprise of each stimulus across different timescales. The results suggest a spatial distribution of regions sensitive to various information regularities according to a temporal hierarchy, which may play a central role in concurrently monitoring the regularity in previous and current events over different timescales to optimize behavioral control in a dynamic environment

Monitoring the past and choosing the future: the prefrontal cortical influences on voluntary action.

Choosing between equivalent response options requires the resolution of ambiguity. One could facilitate such decisions by monitoring previous actions and implementing transient or arbitrary rules to differentiate response options. This would reduce the entropy of chosen actions. We examined voluntary action decisions during magnetoencephalography, identifying the spatiotemporal correlates of stimulus- and choice-entropy. Negative correlations between frontotemporal activity and entropy of past trials were observed after participants' responses, reflecting sequential monitoring of recent events. In contrast, choice entropy correlated negatively with prefrontal activity, before and after participants' response, consistent with transient activation of latent response-sets ahead of a decision and updating the monitor of recent decisions after responding. Individual differences in current choices were related to the strength of the prefrontal signals that reflect monitoring of the statistical regularities in previous events. Together, these results explain individual expressions of voluntary action, through differential engagement of prefrontal areas to guide sequential decisions

Visual perceptual echo reflects learning of regularities in rapid luminance sequences

A novel neural signature of active visual processing has recently been described in the form of the ‘perceptual echo’, in which the cross-correlation between a sequence of randomly fluctuating luminance values and occipital electrophysiological (EEG) signals exhibits a long-lasting periodic (~100ms cycle) reverberation of the input stimulus (VanRullen & Macdonald, 2012). As yet, however, the mechanisms underlying the perceptual echo and its function remain unknown. Reasoning that natural visual signals often contain temporally predictable, though non-periodic features, we hypothesized that the perceptual echo may reflect a periodic process associated with regularity learning. To test this hypothesis, we presented subjects with successive repetitions of a rapid non-periodic luminance sequence, and examined the effects on the perceptual echo, finding that echo amplitude linearly increased with the number of presentations of a given luminance sequence. These data suggest that the perceptual echo reflects a neural signature of regularity learning. Furthermore, when a set of repeated sequences was followed by a sequence with inverted luminance polarities, the echo amplitude decreased to the same level evoked by a novel stimulus sequence. Crucially, when the original stimulus sequence was re-presented, the echo amplitude returned to a level consistent with the number of presentations of this sequence, indicating that the visual system retained sequence specific information, for many seconds, even in the presence of intervening visual input. Altogether, our results reveal a previously undiscovered regularity learning mechanism within the human visual system, reflected by the perceptual echo

Mismatch responses: Probing probabilistic inference in the brain

Sensory signals are governed by statistical regularities and carry valuable information about the unfolding of environmental events. The brain is thought to capitalize on the probabilistic nature of sequential inputs to infer on the underlying (hidden) dynamics driving sensory stimulation. Mis-match responses (MMRs) such as the mismatch negativity (MMN) and the P3 constitute prominent neuronal signatures which are increasingly interpreted as reflecting a mismatch between the current sensory input and the brain’s generative model of incoming stimuli. As such, MMRs might be viewed as signatures of probabilistic inference in the brain and their response dynamics can provide insights into the underlying computational principles. However, given the dominance of the auditory modality in MMR research, the specifics of brain responses to probabilistic sequences across sensory modalities and especially in the somatosensory domain are not well characterized. The work presented here investigates MMRs across the auditory, visual and somatosensory modality by means of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). We designed probabilistic stimulus sequences to elicit and characterize MMRs and employed computational modeling of response dynamics to inspect different aspects of the brain’s generative model of the sensory environment. In the first study, we used a volatile roving stimulus paradigm to elicit somatosensory MMRs and performed single-trial modeling of EEG signals in sensor and source space. Model comparison suggested that responses reflect Bayesian inference based on the estimation of transition probability and limited information integration of the recent past in order to adapt to a changing environment. The results indicated that somatosensory MMRs reflect an initial mismatch between sensory input and model beliefs represented by confidence-corrected surprise (CS) followed by model adjustment dynamics represented by Bayesian surprise (BS). For the second and third study we designed a tri-modal roving stimulus paradigm to delineate modality specific and modality general features of mismatch processing. Computational modeling of EEG signals in study 2 suggested that single-trial dynamics reflect Bayesian inference based on estimation of uni-modal transition probabilities as well as cross-modal conditional dependencies. While early mismatch processing around the MMN tended to reflect CS, later MMRs around the P3 rather reflect BS, in correspondence to the somatosensory study. Finally, the fMRI results of study 3 showed that MMRs are generated by an interaction of modality specific regions in higher order sensory cortices and a modality general fronto-parietal network. Inferior parietal regions in particular were sensitive to expectation violations with respect to the cross-modal contingencies in the stimulus sequences. Overall, our results indicate that MMRs across the senses reflect processes of probabilistic inference in a complex and inherently multi-modal environment.Sensorische Signale sind durch statistische Regularitäten bestimmt und beinhalten wertvolle Informationen über die Entwicklung von Umweltereignissen. Es wird angenommen, dass das Gehirn die Wahrscheinlichkeitseigenschaften sequenzieller Reize nutzt um auf die zugrundeliegenden (verborgenen) Dynamiken zu schließen, welche sensorische Stimulation verursachen. Diskrepanz-Reaktionen ("Mismatch responses"; MMRs) wie die "mismatch negativity" (MMN) und die P3 sind bekannte neuronale Signaturen die vermehrt als Signale einer Diskrepanz zwischen der momentanen sensorischen Einspeisung und dem generativen Modell, welches das Gehirn von den eingehenden Reizen erstellt angesehen werden. Als solche können MMRs als Signaturen von wahrscheinlichkeitsbasierter Inferenz im Gehirn betrachtet werden und ihre Reaktionsdynamiken können Einblicke in die zugrundeliegenden komputationalen Prinzipien geben. Angesichts der Dominanz der auditorischen Modalität in der MMR-Forschung, sind allerdings die spezifischen Eigenschaften von Hirn-Reaktionen auf Wahrscheinlichkeitssequenzen über sensorische Modalitäten hinweg und vor allem in der somatosensorischen Modalität nicht gut charakterisiert. Die hier vorgestellte Arbeit untersucht MMRs über die auditorische, visuelle und somatosensorische Modalität hinweg anhand von Elektroenzephalographie (EEG) und funktioneller Magnetresonanztomographie (fMRT). Wir gestalteten wahrscheinlichkeitsbasierte Reizsequenzen, um MMRs auszulösen und zu charakterisieren und verwendeten komputationale Modellierung der Reaktionsdynamiken, um verschiedene Aspekte des generativen Modells des Gehirns von der sensorischen Umwelt zu untersuchen. In der ersten Studie verwendeten wir ein volatiles "Roving-Stimulus"-Paradigma, um somatosensorische MMRs auszulösen und modellierten die Einzel-Proben der EEG-Signale im sensorischen und Quell-Raum. Modellvergleiche legten nahe, dass die Reaktionen Bayes’sche Inferenz abbilden, basierend auf der Schätzung von Transitionswahrscheinlichkeiten und limitierter Integration von Information der jüngsten Vergangenheit, welche eine Anpassung an Umweltänderungen ermöglicht. Die Ergebnisse legen nahe, dass somatosen-sorische MMRs eine initiale Diskrepanz zwischen sensorischer Einspeisung und Modellüberzeugung reflektieren welche durch "confidence-corrected surprise" (CS) repräsentiert ist, gefolgt von Modelanpassungsdynamiken repräsentiert von "Bayesian surprise" (BS). Für die zweite und dritte Studie haben wir ein Tri-Modales "Roving-Stimulus"-Paradigma gestaltet, um modalitätsspezifische und modalitätsübergreifende Eigenschaften von Diskrepanzprozessierung zu umreißen. Komputationale Modellierung von EEG-Signalen in Studie 2 legte nahe, dass Einzel-Proben Dynamiken Bayes’sche Inferenz abbilden, basierend auf der Schätzung von unimodalen Transitionswahrscheinlichkeiten sowie modalitätsübergreifenden bedingten Abhängigkeiten. Während frühe Diskrepanzprozessierung um die MMN dazu tendierten CS zu reflektieren, so reflektierten spätere MMRs um die P3 eher BS, in Übereinstimmung mit der somatosensorischen Studie. Abschließend zeigten die fMRT-Ergebnisse der Studie 3 dass MMRs durch eine Interaktion von modalitätsspezifischen Regionen in sensorischen Kortizes höherer Ordnung mit einem modalitätsübergreifenden fronto-parietalen Netzwerk generiert werden. Inferior parietale Regionen im Speziellen waren sensitiv gegenüber Erwartungsverstoß in Bezug auf die modalitätsübergreifenden Wahrscheinlichkeiten in den Reizsequenzen. Insgesamt weisen unsere Ergebnisse darauf hin, dass MMRs über die Sinne hinweg Prozesse von wahrscheinlichkeitsbasierter Inferenz in einer komplexen und inhärent multi-modalen Umwelt darstellen

Perceived time and temporal structure: neural entrainment to isochronous stimulation increases duration estimates

Distortions of perceived duration can give crucial insights into the mechanisms that underlie the processing and representation of stimulus timing. One factor that affects duration estimates is the temporal structure of stimuli that fill an interval. For example, regular filling (isochronous interval) leads to an overestimation of perceived duration as compared to irregular filling (anisochronous interval). In the present article, we use electroencephalography (EEG) to investigate the neural basis of this subjective lengthening of perceived duration with isochrony. In a two-interval forced choice task, participants judged which of two intervals lasts longer – one always being isochronous, the other one anisochronous. Response proportions confirm the subjective overestimation of isochronous intervals. At the neural level, isochronous sequences are associated with enhanced pairwise phase consistency (PPC) at the stimulation frequency, reflecting the brain's entrainment to the regular stimulation. The PPC over the entrainment channels is further enhanced for isochronous intervals that are reported to be longer, and the magnitude of this PCC effect correlates with the amount of perceptual bias. Neural entrainment has been proposed as a mechanism of attentional selection, enabling increased neural responsiveness toward stimuli that arrive at an expected point in time. The present results support the proposed relationship between neural response magnitudes and temporal estimates: An increase in neural responsiveness leads to a more pronounced representation of the individual stimuli filling the interval and in turn to a subjective increase in duration

Musical rhythm effects on visual attention are non-rhythmical: Evidence against metrical entrainment

The idea that external rhythms synchronize attention cross-modally has attracted much interest and scientific inquiry. Yet, whether associated attentional modulations are indeed rhythmical in that they spring from and map onto an underlying meter has not been clearly established. Here we tested this idea while addressing the shortcomings of previous work associated with confounding (i) metricality and regularity, (ii) rhythmic and temporal expectations or (iii) global and local temporal effects. We designed sound sequences that varied orthogonally (high/low) in metricality and regularity and presented them as task-irrelevant auditory background in four separate blocks. The participants' task was to detect rare visual targets occurring at a silent metrically aligned or misaligned temporal position. We found that target timing was irrelevant for reaction times and visual event-related potentials. High background regularity and to a lesser extent metricality facilitated target processing across metrically aligned and misaligned positions. Additionally, high regularity modulated auditory background frequencies in the EEG recorded over occipital cortex. We conclude that external rhythms, rather than synchronizing attention cross-modally, confer general, nontemporal benefits. Their predictability conserves processing resources that then benefit stimulus representations in other modalities

Neurons with stereotyped and rapid responses provide a reference frame for relative temporal coding in primate auditory cortex

The precise timing of spikes of cortical neurons relative to stimulus onset carries substantial sensory information. To access this information the sensory systems would need to maintain an internal temporal reference that reflects the precise stimulus timing. Whether and how sensory systems implement such reference frames to decode time-dependent responses, however, remains debated. Studying the encoding of naturalistic sounds in primate (Macaca mulatta) auditory cortex we here investigate potential intrinsic references for decoding temporally precise information. Within the population of recorded neurons, we found one subset responding with stereotyped fast latencies that varied little across trials or stimuli, while the remaining neurons had stimulus-modulated responses with longer and variable latencies. Computational analysis demonstrated that the neurons with stereotyped short latencies constitute an effective temporal reference for relative coding. Using the response onset of a simultaneously recorded stereotyped neuron allowed decoding most of the stimulus information carried by onset latencies and the full spike train of stimulus-modulated neurons. Computational modeling showed that few tens of such stereotyped reference neurons suffice to recover nearly all information that would be available when decoding the same responses relative to the actual stimulus onset. These findings reveal an explicit neural signature of an intrinsic reference for decoding temporal response patterns in the auditory cortex of alert animals. Furthermore, they highlight a role for apparently unselective neurons as an early saliency signal that provides a temporal reference for extracting stimulus information from other neurons

Great expectations: Is there evidence for predictive coding in auditory cortex?

Predictive coding is possibly one of the most influential, comprehensive, and controversial theories of neural function. Whilst proponents praise its explanatory potential, critics object that key tenets of the theory are untested or even untestable. The present article critically examines existing evidence for predictive coding in the auditory modality. Specifically, we identify five key assumptions of the theory and evaluate each in the light of animal, human and modelling studies of auditory pattern processing. For the first two assumptions - that neural responses are shaped by expectations and that these expectations are hierarchically organised - animal and human studies provide compelling evidence. The anticipatory, predictive nature of these expectations also enjoys empirical support, especially from studies on unexpected stimulus omission. However, for the existence of separate error and prediction neurons, a key assumption of the theory, evidence is lacking. More work exists on the proposed oscillatory signatures of predictive coding, and on the relation between attention and precision. However, results on these latter two assumptions are mixed or contradictory. Looking to the future, more collaboration between human and animal studies, aided by model-based analyses will be needed to test specific assumptions and implementations of predictive coding - and, as such, help determine whether this popular grand theory can fulfil its expectations

Monitoring the past and choosing the future: the prefrontal cortical influences on voluntary action

Choosing between equivalent response options requires the resolution of ambiguity. One could facilitate such decisions by monitoring previous actions and implementing transient or arbitrary rules to differentiate response options. This would reduce the entropy of chosen actions. We examined voluntary action decisions during magnetoencephalography, identifying the spatiotemporal correlates of stimulus- and choice-entropy. Negative correlations between frontotemporal activity and entropy of past trials were observed after participants’ responses, reflecting sequential monitoring of recent events. In contrast, choice entropy correlated negatively with prefrontal activity, before and after participants’ response, consistent with transient activation of latent response-sets ahead of a decision and updating the monitor of recent decisions after responding. Individual differences in current choices were related to the strength of the prefrontal signals that reflect monitoring of the statistical regularities in previous events. Together, these results explain individual expressions of voluntary action, through differential engagement of prefrontal areas to guide sequential decisions