Memory in humans and deep language models: Linking hypotheses for model augmentation

The computational complexity of the self-attention mechanism in Transformer models significantly limits their ability to generalize over long temporal durations. Memory-augmentation, or the explicit storing of past information in external memory for subsequent predictions, has become a constructive avenue for mitigating this limitation. We argue that memory-augmented Transformers can benefit substantially from considering insights from the memory literature in humans. We detail an approach to integrating evidence from the human memory system through the specification of cross-domain linking hypotheses. We then provide an empirical demonstration to evaluate the use of surprisal as a linking hypothesis, and further identify the limitations of this approach to inform future research.Comment: 5 figure

Does the prefrontal cortex play an essential role in consciousness? Insights from intracranial electrical stimulation of the human brain

A central debate in philosophy and neuroscience pertains to whether PFC activity plays an essential role in the neural basis of consciousness. Neuroimaging and electrophysiology studies have revealed that the contents of conscious perceptual experience can be successfully decoded from PFC activity, but these findings might be confounded by post- perceptual cognitive processes, such as thinking, reasoning, and decision-making, that are not necessary for con- sciousness. To clarify the involvement of the PFC in consciousness, we present a synthesis of research that has used intracranial electrical stimulation (iES) for the causal modulation of neural activity in the human PFC. This research provides compelling evidence that iES of only certain prefrontal regions (i.e., orbitofrontal cortex and anterior cingu- late cortex) reliably perturbs conscious experience. Conversely, stimulation of anterolateral prefrontal sites, often con- sidered crucial in higher-order and global workspace theories of consciousness, seldom elicits any reportable alterations in consciousness. Furthermore, the wide variety of iES-elicited effects in the PFC (e.g., emotions, thoughts, and olfactory and visual hallucinations) exhibits no clear relation to the immediate environment. Therefore, there is no evidence for the kinds of alterations in ongoing perceptual experience that would be predicted by higher-order or global workspace theories. Nevertheless, effects in the orbitofrontal and anterior cingulate cortices suggest a specific role for these PFC subregions in supporting emotional aspects of conscious experience. Overall, this evidence presents a challenge for higher-order and global workspace theories, which commonly point to the PFC as the basis for con- scious perception based on correlative and possibly confounded information

Thematic relations outperform taxonomic relations in a cued recall task

Prior knowledge has long been known to influence retention of newly experienced information. In particular, known semantic associations across items facilitate subsequent memory for these items, and this effect has been shown to increase with measures of semantic relatedness. In the field of categories and concepts, the processing of taxonomic (e.g., cup-fork, dog-bird) versus thematic (e.g., cup-drink, dog-leash) conceptual relations can be differentiated at the behavioral and neural levels. However, the effects of these distinct conceptual relations on memory remain unresolved. The current study used a stimulus set consisting of thematic, taxonomic, and unrelated noun-noun word pairs, to shed light on this issue. Our results indicate that pairs with thematic relations lead to improved cued memory performance, followed by taxonomic relations, and finally unrelated pairs. This study provides evidence that conceptual relations differ in the extent to which they facilitate cued memory performance

Acoustic features drive event segmentation in speech

International audienceWhile our perceptual experience seems to unfold continuously over time, episodic memory preserves distinct events for storage and recollection. Previous work shows that stability in encoding context serves to temporally bind individual items into sequential composite events. This phenomenon has been almost exclusively studied using visual and spatial memory paradigms. Here we adapt these paradigms to test the role of speaker regularity for event segmentation of complex auditory information. The results of our auditory paradigm replicate the findings in other sensory modalities—finding greater within-event temporal memory for items within speaker-bound events and greater source memory for items at speaker or event transitions. The task we use significantly extends the ecological validity of past paradigms by allowing participants to encode the stimuli without any suggestions on the part of the experimenter. This unique property of our design reveals that, while memory performance is strongly dependent on self-reported mnemonic strategy, behavioral effects associated with event segmentation are robust to changes in mnemonic strategy. Finally, we consider the effect of serial position on segmentation effects during encoding and present a modeling approach to estimate the independent contribution of event segmentation. These findings provide several lines of evidence suggesting that contextual stability in perceptual features drives segmentation during word listening and supports a modality-independent role for mechanisms involved in event segmentation

Thematic relations outperform taxonomic relations in a cued recall task

Prior knowledge has long been known to influence retention of newly experienced information. In particular, known semantic associations across items facilitate subsequent memory for these items, and this effect has been shown to increase with measures of semantic relatedness. In the field of categories and concepts, the processing of taxonomic (e.g., cup-fork, dog-bird) versus thematic (e.g., cup-drink, dog-leash) conceptual relations can be differentiated at the behavioral and neural levels. However, the effects of these distinct conceptual relations on memory remain unresolved. The current study used a stimulus set consisting of thematic, taxonomic, and unrelated noun-noun word pairs, to shed light on this issue. Our results indicate that pairs with thematic relations lead to improved cued memory performance, followed by taxonomic relations, and finally unrelated pairs. This study provides evidence that conceptual relations differ in the extent to which they facilitate cued memory performance

The Naturalistic Free Recall Dataset: four stories, hundreds of participants, and high-fidelity transcriptions

The “Naturalistic Free Recall Dataset" provides transcribed verbal recollections of four spoken narratives, collected from a cohort of 229 participants. Each participant listened to two stories, varying in duration from approximately 8 to 13 minutes, recorded by different speakers. Subsequently, participants were tasked with verbally recalling the narrative content in as much detail as possible and in the correct order. The dataset includes high-fidelity, time-stamped text transcripts of both the original narratives and participants' recollections. To validate the dataset, we apply a previously published method to score memory performance for narrative content. Using this approach, we extend effects traditionally observed in classic list-learning paradigms. Finally, to facilitate the use of these rich data by the community, we offer a brief overview of recent computational methods used to automatically annotate and evaluate memory for narratives. All experimental materials, code and data are publicly available to facilitate new advances in understanding human memory

Data from: Changes in subjective experience elicited by direct stimulation of the human orbitofrontal cortex

Objective: We applied direct cortical stimulation (DCS) to the orbitofrontal cortex (OFC) in neurosurgical patients implanted with intracranial electrodes to probe, with high anatomic precision, the causal link between the OFC and human subjective experience. Methods: We administered 272 instances of DCS at 172 OFC sites in 22 patients with intractable focal epilepsy (from 2011 to 2017), none of whom had seizures originating from the OFC. Results: Our observations revealed a rich variety of affective, olfactory, gustatory, and somatosensory changes in the subjective domain. Elicited experiences were largely neutral or negatively valenced (e.g., aversive smells and tastes, sadness, and anger). Evidence was found for preferential left lateralization of negatively valenced experiences and strong right lateralization of neutral effects. Moreover, most of the elicited effects were observed after stimulation of OFC tissue around the transverse orbital sulcus, and none were seen in the most anterior aspects of the OFC. Conclusions: Our study yielded 3 central findings: first, a dissociation between the “silent” anterior and nonsilent middle/posterior OFC where stimulation clearly elicits changes in subjective experience; second, evidence that the OFC might play a causal role in integrating affect and multimodal sensory experiences; and third, clear evidence for left lateralization of negatively valenced effects. Our findings provide important information for clinicians treating OFC injury or planning OFC resection and scientists seeking to understand the brain basis for the integration of sensation, cognition, and affect