Modeling the Temporal Dynamics of Visual Working Memory

Abstract Visual working memory (VWM) is one of the most crucial parts of the human cognitive system. Research focuses on the apparent limits in the capacity of this system and the reasons for them. So far only a few formal models exist that can account for the temporal dynamics of the amount of information stored in VWM. We propose a combination of the well established theory of visual attention (TVA) with a dynamic memory model, resulting in an iterative, probabilistic framework for VWM. The model includes a consolidation as well as a decay mechanism and employs the strength concept to quantify the availability of a certain memory trace. We evaluate the model on available change detection data

Time processing in visual cortices: How the visual brain encodes and keeps track of time

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT are functionally linked and temporally synchronized during time encoding whereas they are functionally independent and operate serially (V1 followed by V5/MT) while maintaining temporal information in working memory. These data challenge the traditional view of V1 and V5/MT as visuo-spatial features detectors and highlight the functional contribution and the temporal dynamics of these brain regions in the processing of time in millisecond range. The present project resulted in the paper entitled: 'How the visual brain encodes and keeps track of time' by Paolo Salvioni, Lysiann Kalmbach, Micah Murray and Domenica Bueti that is now submitted for publication to the Journal of Neuroscience

Neural oscillations in auditory working memory

The present thesis investigated memory load and memory decay in auditory working memory. Alpha power as a marker for memory load served as the primary indicator for load and decay fluctuations hypothetically reflecting functional inhibition of irrelevant information. Memory load was induced by presenting auditory signals (syllables and pure-tone sequences) in noise because speech-in-noise has been shown before to increase memory load. The aim of the thesis was to assess with magnetoencephalography whether a-priori temporal expectations for the onset-time of a to-be-remembered stimulus reduces memory load. It was reported previously that top-down modulations such as spatial expectations reduce memory load and improve memory performance. However, this effect has neither been investigated with temporal expectations nor in the auditory domain. The present thesis showed that temporal expectations for a syllable in noise reduced memory load. Reduced alpha power during stimulus maintenance as well as improved performance indicated the decrease in memory load. Alpha power effects emerged from the right cingulo-opercular network, presumably reflecting a reduced need for functional inhibition. Critically, symbolic cues induced temporal expectations. This effect could not be replicated for clear speech. However, more implicit temporal expectations based on the passage of time elicited a similar decrease in alpha power for clear speech reflecting reduced memory load. Memory decay was assessed with variable delay phases in an auditory sensory memory task with pure-tone sequences. Similarly to memory performance, alpha power decreased with longer delay phases. Critically, temporal expectations counteracted memory decay and led to more sustained performance as well as alpha power across different delay phases. These alpha-power effects were localized to frontal and parietal attention networks as well as primary auditory and visual sensory areas. This implies the involvement of different brain regions relevant for encoding and maintenance in auditory memory and questions a parsimonious functional inhibition explanation. A correlation of alpha power and behavioral performance underpinned the importance of alpha power for auditory working memory. Altogether, the results of the present thesis provide evidence for a beneficial effect of a-priori temporal expectations for an auditory signal on working memory. Moreover, alpha dynamics were shown to be a distinct marker for the neural efficiency of managing working memory limitations

The temporal dynamics of visual working memory guidance of selective attention

The biased competition model proposes that there is top-down directing of attention to a stimulus matching the contents of working memory (WM), even when the maintenance of a WM representation is detrimental to target relevant performance. Despite many studies elucidating that spatial WM guidance can be present early in the visual processing system, whether visual WM guidance also influences perceptual selection remains poorly understood. Here, we investigated the electrophysiological correlates of early guidance of attention by WM in humans. Participants were required to perform a visual search task while concurrently maintaining object representations in their visual working memory. Behavioral results showed that response times (RTs) were longer when the distractor in the visual search task was held in WM. The earliest WM guidance effect was observed in the P1 component (90-130 ms), with match trials eliciting larger P1 amplitude than mismatch trials. A similar result was also found in the N1 component (160-200 ms). These P1 and N1 effects could not be attributed to bottom-up perceptual priming from the presentation of a memory cue, because there was no significant difference in early ERP component when the cue was merely perceptually identified but not actively held in working memory. Standardized Low Resolution Electrical Tomography Analysis (sLORETA) showed that the early WM guidance occurred in the occipital lobe and the N1-related activation occurred in the parietal gyrus. Time-frequency data suggested that alpha-band event-related spectral perturbation (ERSP) magnitudes increased under the match condition compared with the mismatch condition. In conclusion, the present study suggests that the reappearance of a stimulus held in WM enhanced activity in the occipital area. Subsequently, this initial capture of attention by WM could be inhibited by competing visual inputs through attention re-orientation, reflecting by the alpha-band rhythm

The Spatial and Temporal Construction of Confidence in the Visual Scene

Human subjects can report many items of a cluttered field a few hundred milliseconds after stimulus presentation. This memory decays rapidly and after a second only 3 or 4 items can be stored in working memory. Here we compared the dynamics of objective performance with a measure of subjective report and we observed that 1) Objective performance beyond explicit subjective reports (blindsight) was significantly more pronounced within a short temporal interval and within specific locations of the visual field which were robust across sessions 2) High confidence errors (false beliefs) were largely confined to a small spatial window neighboring the cue. The size of this window did not change in time 3) Subjective confidence showed a moderate but consistent decrease with time, independent of all other experimental factors. Our study allowed us to asses quantitatively the temporal and spatial access to an objective response and to subjective reports

Hypothesis Generation: Temporal Dynamics

In order to bring structure to many of the judgment and decision making problems people encounter, decision makers are often required to generate, from memory, hypotheses explaining their observations. This dissertation focuses on this predecisional process of hypothesis generation which underlies and supports much judgment and decision making behavior. Although we are beginning to understand a great deal about the mechanisms governing the generation and utilization of hypotheses (Thomas, R.P. et al. 2008) more work is needed to fully appreciate how these retrieval, judgment, and choice processes operate in real-world task environments.The present research addresses temporal dynamics underlying hypothesis generation processes. As temporal dynamics are an inevitable precondition for information acquisition and utilization, a full understanding of hypothesis generation processes will remain speculative without systematic examination of the influences of such dynamics. Four experiments examined various pertinent issues in an effort to provide fundamental insights upon which more complete theory can be developed. The influence of information order, information activation in working memory, information agreement, information use (grouped vs. isolated), and working memory allocation were examined. Furthermore, two novel methodologies are forwarded providing unique approaches for assessing the active contents of working memory through time. By exploiting biases in which visual attention is drawn towards items matching the contents of working memory these measures are able to index the current contents of working memory at any given point in time.Several critical findings emerged from this set of experiments. First, results indicate that people tend to weight later information more heavily than earlier information in some, but not all, circumstances. Second, the role of information activation in working memory was implicated as governing its contribution to the hypothesis generation processes wherein more active items contribute more. Third, it was found that the acquisition of information that is inconsistent with hypotheses under consideration causes people to discard these inconsistent hypotheses. This work provides important insights into how internal working memory dynamics interact with external dynamics in shaping the hypothesis generation process that can be used to support the development of a comprehensive computational theory

Explore the Functional Connectivity between Brain Regions during a Chemistry Working Memory Task.

Previous studies have rarely examined how temporal dynamic patterns, event-related coherence, and phase-locking are related to each other. This study assessed reaction-time-sorted spectral perturbation and event-related spectral perturbation in order to examine the temporal dynamic patterns in the frontal midline (F), central parietal (CP), and occipital (O) regions during a chemistry working memory task at theta, alpha, and beta frequencies. Furthermore, the functional connectivity between F-CP, CP-O, and F-O were assessed by component event-related coherence (ERCoh) and component phase-locking (PL) at different frequency bands. In addition, this study examined whether the temporal dynamic patterns are consistent with the functional connectivity patterns across different frequencies and time courses. Component ERCoh/PL measured the interactions between different independent components decomposed from the scalp EEG, mixtures of time courses of activities arising from different brain, and artifactual sources. The results indicate that the O and CP regions' temporal dynamic patterns are similar to each other. Furthermore, pronounced component ERCoh/PL patterns were found to exist between the O and CP regions across each stimulus and probe presentation, in both theta and alpha frequencies. The consistent theta component ERCoh/PL between the F and O regions was found at the first stimulus and after probe presentation. These findings demonstrate that temporal dynamic patterns at different regions are in accordance with the functional connectivity patterns. Such coordinated and robust EEG temporal dynamics and component ERCoh/PL patterns suggest that these brain regions' neurons work together both to induce similar event-related spectral perturbation and to synchronize or desynchronize simultaneously in order to swiftly accomplish a particular goal. The possible mechanisms for such distinct component phase-locking and coherence patterns were also further discussed

Resonant Neural Dynamics of Speech Perception

What is the neural representation of a speech code as it evolves in time? How do listeners integrate temporally distributed phonemic information across hundreds of milliseconds, even backwards in time, into coherent representations of syllables and words? What sorts of brain mechanisms encode the correct temporal order, despite such backwards effects, during speech perception? How does the brain extract rate-invariant properties of variable-rate speech? This article describes an emerging neural model that suggests answers to these questions, while quantitatively simulating challenging data about audition, speech and word recognition. This model includes bottom-up filtering, horizontal competitive, and top-down attentional interactions between a working memory for short-term storage of phonetic items and a list categorization network for grouping sequences of items. The conscious speech and word recognition code is suggested to be a resonant wave of activation across such a network, and a percept of silence is proposed to be a temporal discontinuity in the rate with which such a resonant wave evolves. Properties of these resonant waves can be traced to the brain mechanisms whereby auditory, speech, and language representations are learned in a stable way through time. Because resonances are proposed to control stable learning, the model is called an Adaptive Resonance Theory, or ART, model.Air Force Office of Scientific Research (F49620-01-1-0397); National Science Foundation (IRI-97-20333); Office of Naval Research (N00014-01-1-0624)