Temporal Dynamics of Binocular Disparity Processing with Corticogeniculate Interactions

A neural model is developed to probe how corticogeniculate feedback may contribute to the dynamics of binocular vision. Feedforward and feedback interactions among retinal, lateral geniculate, and cortical simple and complex cells are used to simulate psychophysical and neurobiological data concerning the dynamics of binocular disparity processing, including correct registration of disparity in response to dynamically changing stimuli, binocular summation of weak stimuli, and fusion of anticorrelated stimuli when they are delayed, but not when they are simultaneous. The model exploits dynamic rebounds between opponent ON and OFF cells that are due to imbalances in habituative transmitter gates. It shows how corticogeniculate feedback can carry out a top-down matching process that inhibits incorrect disparity response and reduces persistence of previously correct responses to dynamically changing displays.Air Force Office of scientific Research (F49620-92-J-0499, F49620-92-J-0334, F49620-92-J-0225); Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409, N00014-92-J-4015); Natioanl Science Foundation (IRI-97-20333); Office of Naval Research (N00014-95-0657

Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

The Laminar Architecture of Visual Cortex and Image Processing Technology

The mammalian neocortex is organized into layers which include circuits that form functional columns in cortical maps. A major unsolved problem concerns how bottom-up, top-down, and horizontal interactions are organized within cortical layers to generate adaptive behaviors. This article summarizes a model, called the LAMINART model, of how these interactions help visual cortex to realize: (1) the binding process whereby cortex groups distributed data into coherent object representations; (2) the attentional process whereby cortex selectively processes important events; and (3) the developmental and learning processes whereby cortex stably grows and tunes its circuits to match environmental constraints. Such Laminar Computing completes perceptual groupings that realize the property of Analog Coherence, whereby winning groupings bind together their inducing features without losing their ability to represent analog values of these features. Laminar Computing also efficiently unifies the computational requirements of preattentive filtering and grouping with those of attentional selection. It hereby shows how Adaptive Resonance Theory (ART) principles may be realized within the laminar circuits of neocortex. Applications include boundary segmentation and surface filling-in algorithms for processing Synthetic Aperture Radar images.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); Office of Naval Research (N00014-95-1-0657

Texture Segregation, Surface Representation, and Figure-ground Separation

A widespread view is that most of texture segregation can be accounted for by differences in the spatial frequency content of texture regions. Evidence from both psychophysical and physiological studies indicate, however, that beyond these early filtering stages,there are stages of 3-D boundary segmentation and surface representation that are used to segregate textures. Chromatic segregation of element-arrangement patterns as studied by Beck and colleagues - cannot be completely explained by the filtering mechanisms previously employed to account for achromatic segregation. An element arrangement pattern is composed of two types of elements that are arranged differently in different image regions (e.g., vertically on top and diagonally on bottom). FACADE theory mechanisms that have previously been used to explain data about 3-D vision and figure-ground separation are here used to simulate chromatic texture segregation data, in eluding data with equiluminant elements on dark or light homogenous backgrounds, or backgrounds composed of vertical and horizontal dark or light stripes, or horizontal notched stripes. These data include the fact that segregation of patterns composed of red and blue squares decreases with inereasing luminance of the interspaces. Asymmetric segregation properties under 3-D viewing conditions with the cquiluminant element;; dose or far arc abo simulated. Two key model properties arc a spatial impenetrability property that inhibits boundary grouping across regions with noncolinear texture elements, and a boundary-surface consistency property that uses feedback between boundary and surface representations to eliminate spurious boundary groupings and separate figures from their backgrounds.Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657, ONR N00014-91-J-4100); CNPq/Brazil (520419/96-0); Air Force Office of Scientific Research (F49620-92-J-0334

Cortical Dynamics of 3-D Surface Perception: Binocular and Half-Occluded Scenic Images

Previous models of stereopsis have concentrated on the task of binocularly matching left and right eye primitives uniquely. A disparity smoothness constraint is often invoked to limit the number of possible matches. These approaches neglect the fact that surface discontinuities are both abundant in natural everyday scenes, and provide a useful cue for scene segmentation. da Vinci stereopsis refers to the more general problem of dealing with surface discontinuities and their associated unmatched monocular regions within binocular scenes. This study develops a mathematical realization of a neural network theory of biological vision, called FACADE Theory, that shows how early cortical stereopsis processes are related to later cortical processes of 3-D surface representation. The mathematical model demonstrates through computer simulation how the visual cortex may generate 3-D boundary segmentations and use them to control filling-in of 3-D surface properties in response to visual scenes. Model mechanisms correctly match disparate binocular regions while filling-in monocular regions with the correct depth within a binocularly viewed scene. This achievement required introduction of a new multiscale binocular filter for stereo matching which clarifies how cortical complex cells match image contours of like contrast polarity, while pooling signals from opposite contrast polarities. Competitive interactions among filter cells suggest how false binocular matches and unmatched monocular cues, which contain eye-of-origin information, arc automatically handled across multiple spatial scales. This network also helps to explain data concerning context-sensitive binocular matching. Pooling of signals from even-symmetric and odd-symmctric simple cells at complex cells helps to eliminate spurious activity peaks in matchable signals. Later stages of cortical processing by the blob and interblob streams, including refined concepts of cooperative boundary grouping and reciprocal stream interactions between boundary and surface representations, arc modeled to provide a complete simulation of the da Vinci stereopsis percept.Office of Naval Research (N00014-95-I-0409, N00014-85-1-0657, N00014-92-J-4015, N00014-91-J-4100); Airforce Office of Scientific Research (90-0175); National Science Foundation (IRI-90-00530); The James S. McDonnell Foundation (94-40

Cortical Dynamics of 3-D Vision and Figure-Ground Pop-Out

Air Force Office of Scientific Research (90-0175); Defense Advanced Research Projects Agency (90-0083); Office of Naval Research (N00014-91-J-4100

Linking the Laminar Circuits of Visual Cortex to Visual Perception

A detailed neural model is being developed of how the laminar circuits of visual cortical areas V1 and V2 implement context-sensitive binding processes such as perceptual grouping and attention, and develop and learn in a stable way. The model clarifies how preattentive and attentive perceptual mechanisms are linked within these laminar circuits, notably how bottom-up, top-down, and horizontal cortical connections interact. Laminar circuits allow the responses of visual cortical neurons to be influenced, not only by the stimuli within their classical receptive fields, but also by stimuli in the extra-classical surround. Such context-sensitive visual processing can greatly enhance the analysis of visual scenes, especially those containing targets that are low contrast, partially occluded, or crowded by distractors. Attentional enhancement can selectively propagate along groupings of both real and illusory contours, thereby showing how attention can selectively enhance object representations. Model mechanisms clarify how intracortical and intercortical feedback help to stabilize cortical development and learning. Although feedback plays a key role, fast feedforward processing is possible in response to unambiguous information.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Science Foundation (IRI-97-20333); Office of Naval Research (N00014-95-1-0657

Active Contour Based Segmentation Techniques for Medical Image Analysis

Image processing is a technique which is used to derive information from the images. Segmentation is a section of image processing for the separation or segregation of information from the required target region of the image. There are different techniques used for segmentation of pixels of interest from the image. Active contour is one of the active models in segmentation techniques, which makes use of the energy constraints and forces in the image for separation of region of interest. Active contour defines a separate boundary or curvature for the regions of target object for segmentation. The contour depends on various constraints based on which they are classified into different types such as gradient vector flow, balloon and geometric models. Active contour models are used in various image processing applications specifically in medical image processing. In medical imaging, active contours are used in segmentation of regions from different medical images such as brain CT images, MRI images of different organs, cardiac images and different images of regions in the human body. Active contours can also be used in motion tracking and stereo tracking. Thus, the active contour segmentation is used for the separation of pixels of interest for different image processing

Texture Segregation By Visual Cortex: Perceptual Grouping, Attention, and Learning

A neural model is proposed of how laminar interactions in the visual cortex may learn and recognize object texture and form boundaries. The model brings together five interacting processes: region-based texture classification, contour-based boundary grouping, surface filling-in, spatial attention, and object attention. The model shows how form boundaries can determine regions in which surface filling-in occurs; how surface filling-in interacts with spatial attention to generate a form-fitting distribution of spatial attention, or attentional shroud; how the strongest shroud can inhibit weaker shrouds; and how the winning shroud regulates learning of texture categories, and thus the allocation of object attention. The model can discriminate abutted textures with blurred boundaries and is sensitive to texture boundary attributes like discontinuities in orientation and texture flow curvature as well as to relative orientations of texture elements. The model quantitatively fits a large set of human psychophysical data on orientation-based textures. Object boundar output of the model is compared to computer vision algorithms using a set of human segmented photographic images. The model classifies textures and suppresses noise using a multiple scale oriented filterbank and a distributed Adaptive Resonance Theory (dART) classifier. The matched signal between the bottom-up texture inputs and top-down learned texture categories is utilized by oriented competitive and cooperative grouping processes to generate texture boundaries that control surface filling-in and spatial attention. Topdown modulatory attentional feedback from boundary and surface representations to early filtering stages results in enhanced texture boundaries and more efficient learning of texture within attended surface regions. Surface-based attention also provides a self-supervising training signal for learning new textures. Importance of the surface-based attentional feedback in texture learning and classification is tested using a set of textured images from the Brodatz micro-texture album. Benchmark studies vary from 95.1% to 98.6% with attention, and from 90.6% to 93.2% without attention.Air Force Office of Scientific Research (F49620-01-1-0397, F49620-01-1-0423); National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624