Multiscale sampling model for motion integration

Biologically plausible strategies for visual scene integration across spatial and temporal domains continues to be a challenging topic. The fundamental question we address is whether classical problems in motion integration, such as the aperture problem, can be solved in a model that samples the visual scene at multiple spatial and temporal scales in parallel. We hypothesize that fast interareal connections that allow feedback of information between cortical layers are the key processes that disambiguate motion direction. We developed a neural model showing how the aperture problem can be solved using different spatial sampling scales between LGN, V1 layer 4, V1 layer 6, and area MT. Our results suggest that multiscale sampling, rather than feedback explicitly, is the key process that gives rise to end-stopped cells in V1 and enables area MT to solve the aperture problem without the need for calculating intersecting constraints or crafting intricate patterns of spatiotemporal receptive fields. Furthermore, the model explains why end-stopped cells no longer emerge in the absence of V1 layer 6 activity (Bolz & Gilbert, 1986), why V1 layer 4 cells are significantly more end-stopped than V1 layer 6 cells (Pack, Livingstone, Duffy, & Born, 2003), and how it is possible to have a solution to the aperture problem in area MT with no solution in V1 in the presence of driving feedback. In summary, while much research in the field focuses on how a laminar architecture can give rise to complicated spatiotemporal receptive fields to solve problems in the motion domain, we show that one can reframe motion integration as an emergent property of multiscale sampling achieved concurrently within lamina and across multiple visual areas.This work was supported in part by CELEST, a National Science Foundation Science of Learning Center; NSF SBE-0354378 and OMA-0835976; ONR (N00014-11-1-0535); and AFOSR (FA9550-12-1-0436). (CELEST, a National Science Foundation Science of Learning Center; SBE-0354378 - NSF; OMA-0835976 - NSF; N00014-11-1-0535 - ONR; FA9550-12-1-0436 - AFOSR)Published versio

Eye gaze position before, during and after percept switching of bistable visual stimului

A bistable visual stimulus, such as the Necker Cube or Rubin’s Vase, can be perceived in two different ways which compete against each other and alternate spontaneously. Percept switch rates have been recorded in past psychophysical experiments, but few experiments have measured percept switches while tracking eye movements in human participants. In our study, we use the Eyelink II system to track eye gaze position during spontaneous percept switches of a bistable, structure-from-motion (SFM) cylinder that can be perceived to be rotating clockwise (CW) or counterclockwise (CCW). Participants reported the perceived direction of rotation of the SFM cylinder using key presses. Reliability of participants’ reports was ensured by including unambiguous rotations. Unambiguous rotation was generated by assigning depth using binocular disparity. Gaze positions were measured 50 – 2000 ms before and after key presses. Our pilot data show that during ambiguous cylinder presentation, gaze positions for CW reports clustered to the left half of the cylinder and gaze positions for CCW reports clustered to the right half of the cylinder between 1000ms before and 1500ms after key presses, but no such correlation was found beyond that timeframe. These results suggest that percept switches can be correlated with prior gaze positions for ambiguous stimuli. Our results further suggest that the mechanism underlying percept initiation may be influenced by the visual hemifield where the ambiguous stimulus is located.Published versio

A frontotemporal regional model of Post-Traumatic Stress Disorder

The flashback is a phenomenon in Post-Traumatic Stress Disorder (PTSD) in which traumatic memories are replayed as a reaction to a stimulus. However, the underlying neural mechanisms for this phenomenon are still under investigation. We created a multi-layer model of visual input, entorhinal cortex, hippocampus, prefrontal cortex, basolateral amygdala, and the central nucleus of the amygdala, as a multi-area network to determine how these regions may be distinctively encoding the traumatic events that produce these replays. The current model dynamic shows that highly emotional visual stimuli can be generalized to similar stimuli, more so than events related to neutral stimuli. This result mimics electrophysiological results in the amygdala (Ghosh & Chattarji, 2015). Our network dynamics can be used to create a more nuanced approach to PTSD treatments: it could replicate outcomes of techniques such as Prolonged Exposure (PE) and Eye Movement Desensitization and Reprocessing (EMDR) and improve the spatial and temporal configuration of the technique. Our model characterizes the spatio-temporal aspects of the flashback phenomenon and as such aids in the spatio-temporal fine-tuning of treatments such as EMDR. As a future direction, we can incorporate in the model individual and developmental differences in plasticity in responding to current treatments based on visual stimuli to come up with optimized treatment for each individual affected by PTSD.Published versio

Investigating human memory of self-position using a virtual 3-dimensional visual environment

Knowing one's location in space is crucial for navigation, especially in unknown environments. Recently, systems involved in self-localization for spatial navigation have been found in rodent brains, but the mechanisms for self-localization are not completely understood. Human behavioral experiments can enhance understanding of self-localization systems in human brains. In this study, human observers are placed in a visual virtual reality (VR) to perform a self-positioning task. The VR is created using a 120 Hz projector and a polarizing filter which separates left and right eye images when passed through 3D glasses. Disparity and size cues generate the perception of depth. Participants are placed in a virtual position on the ground for five seconds and then moved by changing their virtual environment around a stable object. Their task is to return to their initial position on the ground with button controls. Principal component analyses show that errors in self-repositioning are not along any particular axes and each participant has a unique pattern of repositioning. Trial durations do not affect accuracy of repositioning and lack of disparity cues increases the standard error of repositioning in all directions. Some participants have lower errors when initial self-positions appear to be on one or the other side of the stable object, suggesting a link between memory of self-position and a preferred point of reference. Future directions of this project are to explore between-subject differences and the effect of stimulus presentation at different frequencies on memory of self-position.Published versionSupporting documentatio

Fast Synchronization of Perpetual Grouping in Laminar Visual Cortical Circuits

Perceptual grouping is well-known to be a fundamental process during visual perception, notably grouping across scenic regions that do not receive contrastive visual inputs. Illusory contours are a classical example of such groupings. Recent psychophysical and neurophysiological evidence have shown that the grouping process can facilitate rapid synchronization of the cells that are bound together by a grouping, even when the grouping must be completed across regions that receive no contrastive inputs. Synchronous grouping can hereby bind together different object parts that may have become desynchronized due to a variety of factors, and can enhance the efficiency of cortical transmission. Neural models of perceptual grouping have clarified how such fast synchronization may occur by using bipole grouping cells, whose predicted properties have been supported by psychophysical, anatomical, and neurophysiological experiments. These models have not, however, incorporated some of the realistic constraints on which groupings in the brain are conditioned, notably the measured spatial extent of long-range interactions in layer 2/3 of a grouping network, and realistic synaptic and axonal signaling delays within and across cells in different cortical layers. This work addresses the question: Can long-range interactions that obey the bipole constraint achieve fast synchronization under realistic anatomical and neurophysiological constraints that initially desynchronize grouping signals? Can the cells that synchronize retain their analog sensitivity to changing input amplitudes? Can the grouping process complete and synchronize illusory contours across gaps in bottom-up inputs? Our simulations show that the answer to these questions is Yes.Office of Naval Research (N00014-01-1-0624); Air Force Office of Scientific Research (F49620-01-1-03097

Does layer 5 of the cortex project to the thalamic reticular nucleus? Implications for core and matrix thalamocortical circuits and sleep spindles

Two distinct thalamocortical (TC) circuits with reciprocal components can be identified in mammals: The core TC circuit, prevalent in sensory thalamus, drives activity focally in the middle cortical layers. In turn, these core thalamic neurons are innervated by small ‘modulatory’ cortical axon terminals from pyramidal neurons in layer 6(L6). The matrix TC circuit, prevalent in high-order thalamus, has a complementary organization: large axon terminals from cortical layer 5(L5) pyramidal neurons drive activity of matrix thalamic neurons that, in turn, innervate broadly and modulate the superficial cortical layers. Situated strategically between the thalamus and cortex, the inhibitory thalamic reticular nucleus (TRN) intercepts all TC communication. Projections from sensory or motor cortices to TRN terminate exclusively as small boutons and originate from L6, akin to core TC circuits. No studies have shown direct projections to TRN from cortical neurons in L5 that participate in matrix circuits. However, in comparison with other cortices, prefrontal cortices issue substantial projections to the thalamus from L5 and send similar driver-like projections to TRN, which terminate as large boutons and could potentially originate from L5. These large prefrontal axon terminals are similar to cortical boutons in the caudate nucleus and the amygdala, which originate mainly from L5. Based on this indirect evidence we tested the hypothesis that cortical L5 neurons project to TRN in matrix networks, by constructing a computational TC circuit that included core and matrix components with an optional cortical L5 to TRN projection (L5-TRN ON/OFF). Based on the features of TC circuits, our model was able to simulate relay and filtering of signals, and could initiate and propagate spindle oscillations. Activation of TRN neurons with L5-TRN ON in our model initiated spindle generation with different powers, depending on the level of cortical feedback and involvement of model core vs. matrix. Our preliminary findings are in agreement with hypotheses that spindles can be classified in core-generated, matrix-generated or mixed types, depending on the pathways involved in their generation, but only if L5-TRN is ON. Simulation results indicate a more diffuse nature of spindles in matrix compared to core, with the mix type showing intermediate properties, suggesting that shifts in the engagement of distinct TRN, core, and matrix circuits may underlie typical sleep spindle generation and states of vigilance. Disruption of TC-TRN circuit balance may underlie seizures, atypical sensory reactivity, and deficits in sleep and attentional gating seen in autism and schizophrenia.Accepted manuscrip

A neural model of border-ownership from kinetic occlusion

Camouflaged animals that have very similar textures to their surroundings are difficult to detect when stationary. However, when an animal moves, humans readily see a figure at a different depth than the background. How do humans perceive a figure breaking camouflage, even though the texture of the figure and its background may be statistically identical in luminance? We present a model that demonstrates how the primate visual system performs figure–ground segregation in extreme cases of breaking camouflage based on motion alone. Border-ownership signals develop as an emergent property in model V2 units whose receptive fields are nearby kinetically defined borders that separate the figure and background. Model simulations support border-ownership as a general mechanism by which the visual system performs figure–ground segregation, despite whether figure–ground boundaries are defined by luminance or motion contrast. The gradient of motion- and luminance-related border-ownership signals explains the perceived depth ordering of the foreground and background surfaces. Our model predicts that V2 neurons, which are sensitive to kinetic edges, are selective to border-ownership (magnocellular B cells). A distinct population of model V2 neurons is selective to border-ownership in figures defined by luminance contrast (parvocellular B cells). B cells in model V2 receive feedback from neurons in V4 and MT with larger receptive fields to bias border-ownership signals toward the figure. We predict that neurons in V4 and MT sensitive to kinetically defined figures play a crucial role in determining whether the foreground surface accretes, deletes, or produces a shearing motion with respect to the background.This work was supported in part by CELEST (NSF SBE-0354378 and OMA-0835976), the Office of Naval Research (ONR N00014-11-1-0535) and Air Force Office of Scientific Research (AFOSR FA9550-12-1-0436). (NSF SBE-0354378 - CELEST; OMA-0835976 - CELEST; ONR N00014-11-1-0535 - Office of Naval Research; AFOSR FA9550-12-1-0436 - Air Force Office of Scientific Research)Published versio

Perception, cognition, and action in hyperspaces: implications on brain plasticity, learning, and cognition

We live in a three-dimensional (3D) spatial world; however, our retinas receive a pair of 2D projections of the 3D environment. By using multiple cues, such as disparity, motion parallax, perspective, our brains can construct 3D representations of the world from the 2D projections on our retinas. These 3D representations underlie our 3D perceptions of the world and are mapped into our motor systems to generate accurate sensorimotor behaviors. Three-dimensional perceptual and sensorimotor capabilities emerge during development: the physiology of the growing baby changes hence necessitating an ongoing re-adaptation of the mapping between 3D sensory representations and the motor coordinates. This adaptation continues in adulthood and is quite general to successfully deal with joint-space changes (longer arms due to growth), skull and eye size changes (and still being able of accurate eye movements), etc. A fundamental question is whether our brains are inherently limited to 3D representations of the environment because we are living in a 3D world, or alternatively, our brains may have the inherent capability and plasticity of representing arbitrary dimensions; however, 3D representations emerge from the fact that our development and learning take place in a 3D world. Here, we review research related to inherent capabilities and limitations of brain plasticity in terms of its spatial representations and discuss whether with appropriate training, humans can build perceptual and sensorimotor representations of spatial 4D environments, and how the presence or lack of ability of a solid and direct 4D representation can reveal underlying neural representations of space.Published versio

Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements

How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.Published versio

Neural dynamics of feedforward and feedback processing in figure-ground segregation

Determining whether a region belongs to the interior or exterior of a shape (figure-ground segregation) is a core competency of the primate brain, yet the underlying mechanisms are not well understood. Many models assume that figure-ground segregation occurs by assembling progressively more complex representations through feedforward connections, with feedback playing only a modulatory role. We present a dynamical model of figure-ground segregation in the primate ventral stream wherein feedback plays a crucial role in disambiguating a figure's interior and exterior. We introduce a processing strategy whereby jitter in RF center locations and variation in RF sizes is exploited to enhance and suppress neural activity inside and outside of figures, respectively. Feedforward projections emanate from units that model cells in V4 known to respond to the curvature of boundary contours (curved contour cells), and feedback projections from units predicted to exist in IT that strategically group neurons with different RF sizes and RF center locations (teardrop cells). Neurons (convex cells) that preferentially respond when centered on a figure dynamically balance feedforward (bottom-up) information and feedback from higher visual areas. The activation is enhanced when an interior portion of a figure is in the RF via feedback from units that detect closure in the boundary contours of a figure. Our model produces maximal activity along the medial axis of well-known figures with and without concavities, and inside algorithmically generated shapes. Our results suggest that the dynamic balancing of feedforward signals with the specific feedback mechanisms proposed by the model is crucial for figure-ground segregation