The Discovery and Interpretation of Evidence Accumulation Stages

To improve the understanding of cognitive processing stages, we combined two prominent traditions in cognitive science: evidence accumulation models and stage discovery methods. While evidence accumulation models have been applied to a wide variety of tasks, they are limited to tasks in which decision-making effects can be attributed to a single processing stage. Here, we propose a new method that first uses machine learning to discover processing stages in EEG data and then applies evidence accumulation models to characterize the duration effects in the identified stages. To evaluate this method, we applied it to a previously published associative recognition task (Application 1) and a previously published random dot motion task with a speed-accuracy trade-off manipulation (Application 2). In both applications, the evidence accumulation models accounted better for the data when we first applied the stage-discovery method, and the resulting parameter estimates where generally in line with psychological theories. In addition, in Application 1 the results shed new light on target-foil effects in associative recognition, while in Application 2 the stage discovery method identified an additional stage in the accuracy-focused condition — challenging standard evidence accumulation accounts. We conclude that the new framework provides a powerful new tool to investigate processing stages

Discovering the brain stages of lexical decision:Behavioral effects originate from a single neural decision process

Lexical decision (LD) – judging whether a sequence of letters constitutes a word – has been widely investigated. In a typical lexical decision task (LDT), participants are asked to respond whether a sequence of letters is an actual word or a nonword. Although behavioral differences between types of words/nonwords have been robustly detected in LDT, there is an ongoing discussion about the exact cognitive processes that underlie the word identification process in this task. To obtain data-driven evidence on the underlying processes, we recorded electroencephalographic (EEG) data and applied a novel machine-learning method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA). In the current study, participants performed an LDT in which we varied the frequency of words (high, low frequency) and “wordlikeness” of non-words (pseudowords, random non-words). The results revealed that models with six processing stages accounted best for the data in all conditions. While most stages were shared, Stage 5 differed between conditions. Together, these results indicate that the differences in word frequency and lexicality effects are driven by a single cognitive processing stage. Based on its latency and topology, we interpret this stage as a Decision process during which participants discriminate between words and nonwords using activated lexical information

Decoding study-independent mind-wandering from EEG using convolutional neural networks

Objective. Mind-wandering is a mental phenomenon where the internal thought process disengages from the external environment periodically. In the current study, we trained EEG classifiers using convolutional neural networks (CNNs) to track mind-wandering across studies. Approach. We transformed the input from raw EEG to band-frequency information (power), single-trial ERP (stERP) patterns, and connectivity matrices between channels (based on inter-site phase clustering). We trained CNN models for each input type from each EEG channel as the input model for the meta-learner. To verify the generalizability, we used leave-N-participant-out cross-validations (N = 6) and tested the meta-learner on the data from an independent study for across-study predictions. Main results. The current results show limited generalizability across participants and tasks. Nevertheless, our meta-learner trained with the stERPs performed the best among the state-of-the-art neural networks. The mapping of each input model to the output of the meta-learner indicates the importance of each EEG channel. Significance. Our study makes the first attempt to train study-independent mind-wandering classifiers. The results indicate that this remains challenging. The stacking neural network design we used allows an easy inspection of channel importance and feature maps.</p

Distinguishing Vigilance Decrement and Low Task Demands from Mind-wandering:A Machine Learning Analysis of EEG

Mind-wandering is a ubiquitous mental phenomenon that is defined as self-generated thought irrelevant to the ongoing task. Mind-wandering tends to occur when people are in a low-vigilance state or when they are performing a very easy task. In the current study, we investigated whether mind-wandering is completely dependent on vigilance and current task demands, or whether it is an independent phenomenon. To this end, we trained support vector machine (SVM) classifiers on EEG data in conditions of low and high vigilance, as well as under conditions of low and high task demands, and subsequently tested those classifiers on participants' self-reported mind-wandering. Participants' momentary mental state was measured by means of intermittent thought probes in which they reported on their current mental state. The results showed that neither the vigilance classifier nor the task demands classifier could predict mind-wandering above-chance level, while a classifier trained on self-reports of mind-wandering was able to do so. This suggests that mind-wandering is a mental state different from low vigilance or performing tasks with low demands—both which could be discriminated from the EEG above chance. Furthermore, we used dipole fitting to source-localize the neural correlates of the most import features in each of the three classifiers, indeed finding a few distinct neural structures between the three phenomena. Our study demonstrates the value of machine-learning classifiers in unveiling patterns in neural data and uncovering the associated neural structures by combining it with an EEG source analysis technique

Memory-related cognitive load effects in an interrupted learning task:A model-based explanation

Background: The Cognitive Load Theory provides a well-established framework for investigating aspects of learning situations that demand learners' working memory resources. However, the interplay of these aspects at the cognitive and neural level is still not fully understood. Method: We developed four computational models in the cognitive architecture ACT-R to clarify underlying memory-related strategies and mechanisms. Our models account for human data of an experiment that required participants to perform a symbol sequence learning task with embedded interruptions. We explored the inclusion of subsymbolic mechanisms to explain these data and used our final model to generate fMRI predictions. Results: The final model indicates a reasonable fit for reaction times and accuracy and links the fMRI predictions to the Cognitive Load Theory. Conclusions: Our work emphasizes the influence of task characteristics and supports a process-related view on cognitive load in instructional scenarios. It further contributes to the discussion of underlying mechanisms at a neural level

A functional spiking-neuron model of activity-silent working memory in humans based on calcium-mediated short-term synaptic plasticity

In this paper, we present a functional spiking-neuron model of human working memory (WM). This model combines neural firing for encoding of information with activity-silent maintenance. While it used to be widely assumed that information in WM is maintained through persistent recurrent activity, recent studies have shown that information can be maintained without persistent firing; instead, information can be stored in activity-silent states. A candidate mechanism underlying this type of storage is short-term synaptic plasticity (STSP), by which the strength of connections between neurons rapidly changes to encode new information. To demonstrate that STSP can lead to functional behavior, we integrated STSP by means of calcium-mediated synaptic facilitation in a large-scale spiking-neuron model and added a decision mechanism. The model was used to simulate a recent study that measured behavior and EEG activity of participants in three delayed-response tasks. In these tasks, one or two visual gratings had to be maintained in WM, and compared to subsequent probes. The original study demonstrated that WM contents and its priority status could be decoded from neural activity elicited by a task-irrelevant stimulus displayed during the activity-silent maintenance period. In support of our model, we show that it can perform these tasks, and that both its behavior as well as its neural representations are in agreement with the human data. We conclude that information in WM can be effectively maintained in activity-silent states by means of calcium-mediated STSP

Mapping working memory retrieval in space and in time:A combined electroencephalography and electrocorticography approach

In this study, we investigated the time course and neural correlates of the retrieval process underlying visual working memory. We made use of a rare dataset in which the same task was recorded using both scalp electroencephalography (EEG) and Electrocorticography (ECoG), respectively. This allowed us to examine with great spatial and temporal detail how the retrieval process works, and in particular how the medial temporal lobe (MTL) is involved. In each trial, participants judged whether a probe face had been among a set of recently studied faces. With a method that combines hidden semi-Markov models and multivariate pattern analysis, the neural signal was decomposed into a sequence of latent cognitive stages with information about their durations on a trial-by-trial basis. Analyzed separately, EEG and ECoG data yielded converging results on discovered stages and their interpretation, which reflected 1) a brief pre-attention stage, 2) encoding the stimulus, 3) retrieving the studied set, and 4) making a decision. Combining these stages with the high spatial resolution of ECoG suggested that activity in the temporal cortex reflected item familiarity in the retrieval stage; and that once retrieval is complete, there is active maintenance of the studied face set in the decision stage in the MTL. During this same period, the frontal cortex guides the decision by means of theta coupling with the MTL. These observations generalize previous findings on the role of MTL theta from long-term memory tasks to short-term memory tasks

Driving and Multitasking:The Good, the Bad, and the Dangerous

Previous research has shown that multitasking can have a positive or a negative influence on driving performance. The aim of this study was to determine how the interaction between driving circumstances and cognitive requirements of secondary tasks affect a driver’s ability to control a car. We created a driving simulator paradigm where participants had to perform one of two scenarios: one with no traffic in the driver’s lane, and one with substantial traffic in both lanes, some of which had to be overtaken. Four different secondary task conditions were combined with these driving scenarios. In both driving scenarios, using a tablet resulted in the worst, most dangerous, performance, while passively listening to the radio or answering questions for a radio quiz led to the best driving performance. Interestingly, driving as a single task did not produce better performance than driving in combination with one of the radio tasks, and even tended to be slightly worse. These results suggest that drivers switch to internally focused secondary tasks when nothing else is available during monotonous or repetitive driving environments. This mind wandering potentially has a stronger interference effect with driving than non-visual secondary tasks

Thalamic bursts modulate cortical synchrony locally to switch between states of global functional connectivity in a cognitive task

Performing a cognitive task requires going through a sequence of functionally diverse stages. Although it is typically assumed that these stages are characterized by distinct states of cortical synchrony that are triggered by sub-cortical events, little reported evidence supports this hypothesis. To test this hypothesis, we first identified cognitive stages in single-trial MEG data of an associative recognition task, showing with a novel method that each stage begins with local modulations of synchrony followed by a state of directed functional connectivity. Second, we developed the first whole-brain model that can simulate cortical synchrony throughout a task. The model suggests that the observed synchrony is caused by thalamocortical bursts at the onset of each stage, targeted at cortical synapses and interacting with the structural anatomical connectivity. These findings confirm that cognitive stages are defined by distinct states of cortical synchrony and explains the network-level mechanisms necessary for reaching stage-dependent synchrony states