Online Continual Learning on Sequences

Online continual learning (OCL) refers to the ability of a system to learn over time from a continuous stream of data without having to revisit previously encountered training samples. Learning continually in a single data pass is crucial for agents and robots operating in changing environments and required to acquire, fine-tune, and transfer increasingly complex representations from non-i.i.d. input distributions. Machine learning models that address OCL must alleviate \textit{catastrophic forgetting} in which hidden representations are disrupted or completely overwritten when learning from streams of novel input. In this chapter, we summarize and discuss recent deep learning models that address OCL on sequential input through the use (and combination) of synaptic regularization, structural plasticity, and experience replay. Different implementations of replay have been proposed that alleviate catastrophic forgetting in connectionists architectures via the re-occurrence of (latent representations of) input sequences and that functionally resemble mechanisms of hippocampal replay in the mammalian brain. Empirical evidence shows that architectures endowed with experience replay typically outperform architectures without in (online) incremental learning tasks.Comment: L. Oneto et al. (eds.), Recent Trends in Learning From Data, Studies in Computational Intelligence 89

Principal component analysis of ensemble recordings reveals cell assemblies at high temporal resolution

Simultaneous recordings of many single neurons reveals unique insights into network processing spanning the timescale from single spikes to global oscillations. Neurons dynamically self-organize in subgroups of coactivated elements referred to as cell assemblies. Furthermore, these cell assemblies are reactivated, or replayed, preferentially during subsequent rest or sleep episodes, a proposed mechanism for memory trace consolidation. Here we employ Principal Component Analysis to isolate such patterns of neural activity. In addition, a measure is developed to quantify the similarity of instantaneous activity with a template pattern, and we derive theoretical distributions for the null hypothesis of no correlation between spike trains, allowing one to evaluate the statistical significance of instantaneous coactivations. Hence, when applied in an epoch different from the one where the patterns were identified, (e.g. subsequent sleep) this measure allows to identify times and intensities of reactivation. The distribution of this measure provides information on the dynamics of reactivation events: in sleep these occur as transients rather than as a continuous process

An Approach for Reliably Investigating Hippocampal Sharp Wave-Ripples In Vitro

Among the various hippocampal network patterns, sharp wave-ripples (SPW-R) are currently the mechanistically least understood. Although accurate information on synaptic interactions between the participating neurons is essential for comprehensive understanding of the network function during complex activities like SPW-R, such knowledge is currently notably scarce. counterpart. We show that slice storage in the interface chamber close to physiological temperature is the required condition to preserve network integrity that is necessary for the generation of SPW-R. Moreover, we demonstrate the utility of our method for studying synaptic and network properties of SPW-R, using electrophysiological and imaging methods that can only be applied in the submerged system.The approach presented here demonstrates a reliable and experimentally simple strategy for studying hippocampal sharp wave-ripples. Given its utility and easy application we expect our model to foster the generation of new insight into the network physiology underlying SPW-R

Impact of cognitive stimulation on ripples within human epileptic and non-epileptic hippocampus

Background: Until now there has been no way of distinguishing between physiological and epileptic hippocampal ripples in intracranial recordings. In the present study we addressed this by investigating the effect of cognitive stimulation on interictal high frequency oscillations in the ripple range (80-250 Hz) within epileptic (EH) and non-epileptic hippocampus (NH). Methods: We analyzed depth EEG recordings in 10 patients with intractable epilepsy, in whom hippocampal activity was recorded initially during quiet wakefulness and subsequently during a simple cognitive task. Using automated detection of ripples based on amplitude of the power envelope, we analyzed ripple rate (RR) in the cognitive and resting period, within EH and NH. Results: Compared to quiet wakefulness we observed a significant reduction of RR during cognitive stimulation in EH, while it remained statistically marginal in NH. Further, we investigated the direct impact of cognitive stimuli on ripples (i.e. immediately post-stimulus), which showed a transient statistically significant suppression of ripples in the first second after stimuli onset in NH only. Conclusion: Our results point to a differential reactivity of ripples within EH and NH to cognitive stimulation

At clinically relevant concentrations the anaesthetic/amnesic thiopental but not the anticonvulsant phenobarbital interferes with hippocampal sharp wave-ripple complexes

<p>Abstract</p> <p>Background</p> <p>Many sedative agents, including anesthetics, produce explicit memory impairment by largely unknown mechanisms. Sharp-wave ripple (SPW-R) complexes are network activity thought to represent the neuronal substrate for information transfer from the hippocampal to neocortical circuits, contributing to the explicit memory consolidation. In this study we examined and compared the actions of two barbiturates with distinct amnesic actions, the general anesthetic thiopental and the anticonvulsant phenobarbital, on in vitro SPW-R activity.</p> <p>Results</p> <p>Using an in vitro model of SPW-R activity we found that thiopental (50–200 μM) significantly and concentration-dependently reduced the incidence of SPW-R events (it increased the inter-event period by 70–430 %). At the concentration of 25 μM, which clinically produces mild sedation and explicit memory impairment, thiopental significantly reduced the quantity of ripple oscillation (it reduced the number of ripples and the duration of ripple episodes by 20 ± 5%, n = 12, <it>P </it>< 0.01), and suppressed the rhythmicity of SPWs by 43 ± 15% (n = 6, <it>P </it>< 0.05). The drug disrupted the synchrony of SPWs within the CA1 region at 50 μM (by 19 ± 12%; n = 5, <it>P </it>< 0.05). Similar effects of thiopental were observed at higher concentrations. Thiopental did not affect the frequency of ripple oscillation at any of the concentrations tested (10–200 μM). Furthermore, the drug significantly prolonged single SPWs at concentrations ≥50 μM (it increased the half-width and the duration of SPWs by 35–90 %). Thiopental did not affect evoked excitatory synaptic potentials and its results on SPW-R complexes were also observed under blockade of NMDA receptors. Phenobarbital significantly accelerated SPWs at 50 and 100 μM whereas it reduced their rate at 200 and 400 μM. Furthermore, it significantly prolonged SPWs, reduced their synchrony and reduced the quantity of ripples only at the clinically very high concentration of 400 μM, reported to affect memory.</p> <p>Conclusion</p> <p>We hypothesize that thiopental, by interfering with SPW-R activity, through enhancement of the GABA_Areceptor-mediated transmission, affects memory processes which involve hippocampal circuit activation. The quantity but not the frequency of ripple oscillation was affected by the drug.</p

Hippocampal state-dependent behavioral reflex to an identical sensory input in rats.

We examined the local field potential of the hippocampus to monitor brain states during a conditional discrimination task, in order to elucidate the relationship between ongoing brain states and a conditioned motor reflex. Five 10-week-old Wistar/ST male rats underwent a serial feature positive conditional discrimination task in eyeblink conditioning using a preceding light stimulus as a conditional cue for reinforced trials. In this task, a 2-s light stimulus signaled that the following 350-ms tone (conditioned stimulus) was reinforced with a co-terminating 100-ms periorbital electrical shock. The interval between the end of conditional cue and the onset of the conditioned stimulus was 4±1 s. The conditioned stimulus was not reinforced when the light was not presented. Animals successfully utilized the light stimulus as a conditional cue to drive differential responses to the identical conditioned stimulus. We found that presentation of the conditional cue elicited hippocampal theta oscillations, which persisted during the interval of conditional cue and the conditioned stimulus. Moreover, expression of the conditioned response to the tone (conditioned stimulus) was correlated with the appearance of theta oscillations immediately before the conditioned stimulus. These data support hippocampal involvement in the network underlying a conditional discrimination task in eyeblink conditioning. They also suggest that the preceding hippocampal activity can determine information processing of the tone stimulus in the cerebellum and its associated circuits

Repeated Labilization-Reconsolidation Processes Strengthen Declarative Memory in Humans

The idea that memories are immutable after consolidation has been challenged. Several reports have shown that after the presentation of a specific reminder, reactivated old memories become labile and again susceptible to amnesic agents. Such vulnerability diminishes with the progress of time and implies a re-stabilization phase, usually referred to as reconsolidation. To date, the main findings describe the mechanisms associated with the labilization-reconsolidation process, but little is known about its functionality from a biological standpoint. Indeed, two functions have been proposed. One suggests that destabilization of the original memory after the reminder allows the integration of new information into the background of the original memory (memory updating), and the other suggests that the labilization-reconsolidation process strengthens the original memory (memory strengthening). We have previously reported the reconsolidation of human declarative memories, demonstrating memory updating in the framework of reconsolidation. Here we deal with the strengthening function attributed to the reconsolidation process. We triggered labilization-reconsolidation processes successively by repeated presentations of the proper reminder. Participants learned an association between five cue-syllables and their respective response-syllables. Twenty-four hours later, the paired-associate verbal memory was labilized by exposing the subjects to one, two or four reminders. The List-memory was evaluated on Day 3 showing that the memory was improved when at least a second reminder was presented in the time window of the first labilization-reconsolidation process prompted by the earlier reminder. However, the improvement effect was revealed on Day 3, only when at least two reminders were presented on Day2 and not as a consequence of only retrieval. Therefore, we propose central concepts for the reconsolidation process, emphasizing its biological role and the parametrical constrains for this function to be operative

The Neural Representation of Prospective Choice during Spatial Planning and Decisions

We are remarkably adept at inferring the consequences of our actions, yet the neuronal mechanisms that allow us to plan a sequence of novel choices remain unclear. We used functional magnetic resonance imaging (fMRI) to investigate how the human brain plans the shortest path to a goal in novel mazes with one (shallow maze) or two (deep maze) choice points. We observed two distinct anterior prefrontal responses to demanding choices at the second choice point: one in rostrodorsal medial prefrontal cortex (rd-mPFC)/superior frontal gyrus (SFG) that was also sensitive to (deactivated by) demanding initial choices and another in lateral frontopolar cortex (lFPC), which was only engaged by demanding choices at the second choice point. Furthermore, we identified hippocampal responses during planning that correlated with subsequent choice accuracy and response time, particularly in mazes affording sequential choices. Psychophysiological interaction (PPI) analyses showed that coupling between the hippocampus and rd-mPFC increases during sequential (deep versus shallow) planning and is higher before correct versus incorrect choices. In short, using a naturalistic spatial planning paradigm, we reveal how the human brain represents sequential choices during planning without extensive training. Our data highlight a network centred on the cortical midline and hippocampus that allows us to make prospective choices while maintaining initial choices during planning in novel environments

Finding the engram.

Many attempts have been made to localize the physical trace of a memory, or engram, in the brain. However, until recently, engrams have remained largely elusive. In this Review, we develop four defining criteria that enable us to critically assess the recent progress that has been made towards finding the engram. Recent \u27capture\u27 studies use novel approaches to tag populations of neurons that are active during memory encoding, thereby allowing these engram-associated neurons to be manipulated at later times. We propose that findings from these capture studies represent considerable progress in allowing us to observe, erase and express the engram