Parsing a mental program: fixation-related brain signatures of unitary operations and routines in natural visual search

Visual search involves a sequence or routine of unitary operations (i.e. fixations) embedded in a larger mental global program. The process can indeed be seen as a program based on a while loop (while the target is not found), a conditional construct (whether the target is matched or not based on specific recognition algorithms) and a decision making step to determine the position of the next searched location based on existent evidence. Recent developments in our ability to co-register brain scalp potentials (EEG) during free eye movements has allowed investigating brain responses related to fixations (fixation-Related Potentials; fERPs), including the identification of sensory and cognitive local EEG components linked to individual fixations. However, the way in which the mental program guiding the search unfolds has not yet been investigated. We performed an EEG and eye tracking co-registration experiment in which participants searched for a target face in natural images of crowds. Here we show how unitary steps of the program are encoded by specific local target detection signatures and how the positioning of each unitary operation within the global search program can be pinpointed by changes in the EEG signal amplitude as well as the signal power in different frequency bands. By simultaneously studying brain signatures of unitary operations and those occurring during the sequence of fixations, our study sheds light into how local and global properties are combined in implementing visual routines in natural tasks

Exploring the Common Mechanisms of Motion-Based Visual Prediction

Human vision supports prediction for moving stimuli. Here we take an individual differences approach to investigate whether there could be a common processing rate for motion-based visual prediction across diverse motion phenomena. Motion Induced Spatial Conflict (MISC) refers to an incongruity arising from two edges of a combined stimulus, moving rigidly, but with different apparent speeds. This discrepancy induces an illusory jitter that has been attributed to conflict within a motion prediction mechanism. Its apparent frequency has been shown to correlate with the frequency of alpha oscillations in the brain. We asked what other psychophysical measures might correlate positively with MISC frequency. We measured the correlation between MISC jitter frequency and another three measures that might be linked to motion-based spatial prediction. We demonstrate that the illusory jitter frequency in MISC correlates significantly with the accrual rate of the Motion Induced Position Shift (MIPS) effect - the well-established observation that a carrier movement in a static envelope of a Gabor target leads to an apparent position shift of the envelope in the direction of motion. We did not observe significant correlations with the other two measures – the Adaptation Induced Spatial Shift accrual rate (AISS) and the Smooth Motion Threshold (SMT). These results suggest a shared perceptual rate between MISC and MIPS, implying a common periodic mechanism for motion-based visual prediction

STDP Forms Associations between Memory Traces in Networks of Spiking Neurons

Memory traces and associations between them are fundamental for cognitive brain function. Neuron recordings suggest that distributed assemblies of neurons in the brain serve as memory traces for spatial information, real-world items, and concepts. However, there is conflicting evidence regarding neural codes for associated memory traces. Some studies suggest the emergence of overlaps between assemblies during an association, while others suggest that the assemblies themselves remain largely unchanged and new assemblies emerge as neural codes for associated memory items. Here we study the emergence of neural codes for associated memory items in a generic computational model of recurrent networks of spiking neurons with a data-constrained rule for spike-timing-dependent plasticity. The model depends critically on 2 parameters, which control the excitability of neurons and the scale of initial synaptic weights. By modifying these 2 parameters, the model can reproduce both experimental data from the human brain on the fast formation of associations through emergent overlaps between assemblies, and rodent data where new neurons are recruited to encode the associated memories. Hence, our findings suggest that the brain can use both of these 2 neural codes for associations, and dynamically switch between them during consolidation

Looking for a face in the crowd: Fixation-related potentials in an eye-movement visual search task

Despite the compelling contribution of the study of event related potentials (ERPs) and eye movements to cognitive neuroscience, these two approaches have largely evolved independently. We designed an eye-movement visual search paradigm that allowed us to concurrently record EEG and eye movements while subjects were asked to find a hidden target face in a crowded scene with distractor faces. Fixation event-related potentials (fERPs) to target and distractor stimuli showed the emergence of robust sensory components associated with the perception of stimuli and cognitive components associated with the detection of target faces. We compared those components with the ones obtained in a control task at fixation: qualitative similarities as well as differences in terms of scalp topography and latency emerged between the two. By using single trial analyses, fixations to target and distractors could be decoded from the EEG signals above chance level in 11 out of 12 subjects. Our results show that EEG signatures related to cognitive behavior develop across spatially unconstrained exploration of natural scenes and provide a first step towards understanding the mechanisms of target detection during natural search.Fil: Kaunitz, Lisandro N.. University of Leicester; Reino UnidoFil: Kamienkowski, Juan Esteban. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Física. Laboratorio de Neurociencia Integrativa; Argentina. Universidad Diego Portales; Chile;Fil: Varatharajah, Alexander. University of Leicester; Reino UnidoFil: Sigman, Mariano. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Bahía Blanca. Instituto de Física del Sur; Argentina. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Física. Laboratorio de Neurociencia Integrativa; ArgentinaFil: Quian Quiroga, Rodrigo. University of Leicester; Reino UnidoFil: Ison, Matias Julian. University of Leicester; Reino Unid

Scene-selective coding by single neurons in the human parahippocampal cortex

Imaging, electrophysiological, and lesion studies have shown a relationship between the parahippocampal cortex (PHC) and the processing of spatial scenes. Our present knowledge of PHC, however, is restricted to the macroscopic properties and dynamics of bulk tissue; the behavior and selectivity of single parahippocampal neurons remains largely unknown. In this study, we analyzed responses from 630 parahippocampal neurons in 24 neurosurgical patients during visual stimulus presentation. We found a spatially clustered subpopulation of scene-selective units with an associated event-related field potential. These units form a population code that is more distributed for scenes than for other stimulus categories, and less sparse than elsewhere in the medial temporal lobe. Our electrophysiological findings provide insight into how individual units give rise to the population response observed with functional imaging in the parahippocampal place area

Selectivity of pyramidal cells and interneurons in the human medial temporal lobe

Neurons in the medial temporal lobe (MTL) respond selectively to pictures of specific individuals, objects, and places. However, the underlying mechanisms leading to such degree of stimulus selectivity are largely unknown. A necessary step to move forward in this direction involves the identification and characterization of the different neuron types present in MTL circuitry. We show that putative principal cells recorded in vivo from the human MTL are more selective than putative interneurons. Furthermore, we report that putative hippocampal pyramidal cells exhibit the highest degree of selectivity within the MTL, reflecting the hierarchical processing of visual information. We interpret these differences in selectivity as a plausible mechanism for generating sparse responses

Specific responses of human hippocampal neurons are associated with better memory

A population of human hippocampal neurons has shown responses to individual concepts (e.g., Jennifer Aniston) that generalize to different instances of the concept. However, recordings from the rodent hippocampus suggest an important function of these neurons is their ability to discriminate overlapping representations, or pattern separate, a process that may facilitate discrimination of similar events for successful memory. In the current study, we explored whether human hippocampal neurons can also demonstrate the ability to discriminate between overlapping representations and whether this selectivity could be directly related to memory performance. We show that among medial temporal lobe (MTL) neurons, certain populations of neurons are selective for a previously studied (target) image in that they show a significant decrease in firing rate to very similar (lure) images. We found that a greater proportion of these neurons can be found in the hippocampus compared with other MTL regions, and that memory for individual items is correlated to the degree of selectivity of hippocampal neurons responsive to those items. Moreover, a greater proportion of hippocampal neurons showed selective firing for target images in good compared with poor performers, with overall memory performance correlated with hippocampal selectivity. In contrast, selectivity in other MTL regions was not associated with memory performance. These findings show that a substantial proportion of human hippocampal neurons encode specific memories that support the discrimination of overlapping representations. These results also provide previously unidentified evidence consistent with a unique role of the human hippocampus in orthogonalization of representations in declarative memory

Decoding of human identity by computer vision and neuronal vision

Extracting meaning from a dynamic and variable flow of incoming information is a major goal of both natural and artificial intelligence. Computer vision (CV) guided by deep learning (DL) has made significant strides in recognizing a specific identity despite highly variable attributes. This is the same challenge faced by the nervous system and partially addressed by the concept cells—neurons exhibiting selective firing in response to specific persons/places, described in the human medial temporal lobe (MTL) ⁠. Yet, access to neurons representing a particular concept is limited due to these neurons’ sparse coding. It is conceivable, however, that the information required for such decoding is present in relatively small neuronal populations. To evaluate how well neuronal populations encode identity information in natural settings, we recorded neuronal activity from multiple brain regions of nine neurosurgical epilepsy patients implanted with depth electrodes, while the subjects watched an episode of the TV series “24”. First, we devised a minimally supervised CV algorithm (with comparable performance against manually-labeled data) to detect the most prevalent characters (above 1% overall appearance) in each frame. Next, we implemented DL models that used the time-varying population neural data as inputs and decoded the visual presence of the four main characters throughout the episode. This methodology allowed us to compare “computer vision” with “neuronal vision”—footprints associated with each character present in the activity of a subset of neurons—and identify the brain regions that contributed to this decoding process. We then tested the DL models during a recognition memory task following movie viewing where subjects were asked to recognize clip segments from the presented episode. DL model activations were not only modulated by the presence of the corresponding characters but also by participants’ subjective memory of whether they had seen the clip segment, and by the associative strengths of the characters in the narrative plot. The described approach can offer novel ways to probe the representation of concepts in time-evolving dynamic behavioral tasks. Further, the results suggest that the information required to robustly decode concepts is present in the population activity of only tens of neurons even in brain regions beyond MTL

A category-specific response to animals in the right human amygdala

The amygdala is important in emotion, but it remains unknown whether it is specialized for certain stimulus categories. We analyzed responses recorded from 489 single neurons in the amygdalae of 41 neurosurgical patients and found a categorical selectivity for pictures of animals in the right amygdala. This selectivity appeared to be independent of emotional valence or arousal and may reflect the importance that animals held throughout our evolutionary past