A robust contour detection operator with combined push-pull inhibition and surround suppression

Contour detection is a salient operation in many computer vision applications as it extracts features that are important for distinguishing objects in scenes. It is believed to be a primary role of simple cells in visual cortex of the mammalian brain. Many of such cells receive push-pull inhibition or surround suppression. We propose a computational model that exhibits a combination of these two phenomena. It is based on two existing models, which have been proven to be very effective for contour detection. In particular, we introduce a brain-inspired contour operator that combines push-pull and surround inhibition. It turns out that this combination results in a more effective contour detector, which suppresses texture while keeping the strongest responses to lines and edges, when compared to existing models. The proposed model consists of a Combination of Receptive Field (or CORF) model with push-pull inhibition, extended with surround suppression. We demonstrate the effectiveness of the proposed approach on the RuG and Berkeley benchmark data sets of 40 and 500 images, respectively. The proposed push-pull CORF operator with surround suppression outperforms the one without suppression with high statistical significance

Improved line/edge detection and visual reconstruction

Lines and edges provide important information for object categorization and recognition. In addition, one brightness model is based on a symbolic interpretation of the cortical multi-scale line/edge representation. In this paper we present an improved scheme for line/edge extraction from simple and complex cells and we illustrate the multi-scale representation. This representation can be used for visual reconstruction, but also for nonphotorealistic rendering. Together with keypoints and a new model of disparity estimation, a 3D wireframe representation of e.g. faces can be obtained in the future

A Push-Pull CORF Model of a Simple Cell with Antiphase Inhibition Improves SNR and Contour Detection

We propose a computational model of a simple cell with push-pull inhibition, a property that is observed in many real simple cells. It is based on an existing model called Combination of Receptive Fields or CORF for brevity. A CORF model uses as afferent inputs the responses of model LGN cells with appropriately aligned center-surround receptive fields, and combines their output with a weighted geometric mean. The output of the proposed model simple cell with push-pull inhibition, which we call push-pull CORF, is computed as the response of a CORF model cell that is selective for a stimulus with preferred orientation and preferred contrast minus a fraction of the response of a CORF model cell that responds to the same stimulus but of opposite contrast. We demonstrate that the proposed push-pull CORF model improves signal-to-noise ratio (SNR) and achieves further properties that are observed in real simple cells, namely separability of spatial frequency and orientation as well as contrast-dependent changes in spatial frequency tuning. We also demonstrate the effectiveness of the proposed push-pull CORF model in contour detection, which is believed to be the primary biological role of simple cells. We use the RuG (40 images) and Berkeley (500 images) benchmark data sets of images with natural scenes and show that the proposed model outperforms, with very high statistical significance, the basic CORF model without inhibition, Gabor-based models with isotropic surround inhibition, and the Canny edge detector. The push-pull CORF model that we propose is a contribution to a better understanding of how visual information is processed in the brain as it provides the ability to reproduce a wider range of properties exhibited by real simple cells. As a result of push-pull inhibition a CORF model exhibits an improved SNR, which is the reason for a more effective contour detection.</p

Artistic rendering of the visual cortex

In this paper we explain the processing in the first layers of the visual cortex by simple, complex and endstopped cells, plus grouping cells for line, edge, keypoint and saliency detection. Three visualisations are presented: (a) an integrated scheme that shows activities of simple, complex and end-stopped cells, (b) artistic combinations of selected activity maps that give an impression of global image structure and/or local detail, and (c) NPR on the basis of a 2D brightness model. The cortical image representations offer many possibilities for non-photorealistic rendering

Beyond the classic receptive field: the effect of contextual stimuli

Following the pioneering studies of the receptive field (RF), the concept gained further significance for visual perception by the discovery of input effects from beyond the classical RF. These studies demonstrated that neuronal responses could be modulated by stimuli outside their RFs, consistent with the perception of induced brightness, color, orientation, and motion. Lesion scotomata are similarly modulated perceptually from the surround by RFs that have migrated from the interior to the outer edge of the scotoma and in this way provide filling-in of the void. Large RFs are advantageous to this task. In higher visual areas, such as the middle temporal and inferotemporal lobe, RFs increase in size and lose most of their retinotopic organization while encoding increasingly complex features. Whereas lowerlevel RFs mediate perceptual filling-in, contour integration, and figure–ground segregation, RFs at higher levels serve the perception of grouping by common fate, biological motion, and other biologically relevant stimuli, such as faces. Studies in alert monkeys while freely viewing natural scenes showed that classical and nonclassical RFs cooperate in forming representations of the visual world. Today, our understanding of the mechanisms underlying the RF is undergoing a quantum leap. What had started out as a hierarchical feedforward concept for simple stimuli, such as spots, lines, and bars, now refers to mechanisms involving ascending, descending, and lateral signal flow. By extension of the bottom-up paradigm, RFs are nowadays understood as adaptive processors, enabling the predictive coding of complex scenes. Top-down effects guiding attention and tuned to task-relevant information complement the bottom-up analysis

Dynamics of contextual modulation of perceived shape in human vision

In biological vision, contextual modulation refers to the influence of a surround pattern on either the perception of, or the neural responses to, a target pattern. One studied form of contextual modulation deals with the effect of a surround texture on the perceived shape of a contour, in the context of the phenomenon known as the shape aftereffect. In the shape aftereffect, prolonged viewing, or adaptation to a particular contour’s shape causes a shift in the perceived shape of a subsequently viewed contour. Shape aftereffects are suppressed when the adaptor contour is surrounded by a texture of similarly-shaped contours, a surprising result given that the surround contours are all potential adaptors. Here we determine the motion and temporal properties of this form of contextual modulation. We varied the relative motion directions, speeds and temporal phases between the central adaptor contour and the surround texture and measured for each manipulation the degree to which the shape aftereffect was suppressed. Results indicate that contextual modulation of shape processing is selective to motion direction, temporal frequency and temporal phase. These selectivities are consistent with one aim of vision being to segregate contours that define objects from those that form textured surfaces

Feedback and surround modulated boundary detection

Altres ajuts: CERCA Programme/Generalitat de CatalunyaEdges are key components of any visual scene to the extent that we can recognise objects merely by their silhouettes. The human visual system captures edge information through neurons in the visual cortex that are sensitive to both intensity discontinuities and particular orientations. The "classical approach" assumes that these cells are only responsive to the stimulus present within their receptive fields, however, recent studies demonstrate that surrounding regions and inter-areal feedback connections influence their responses significantly. In this work we propose a biologically-inspired edge detection model in which orientation selective neurons are represented through the first derivative of a Gaussian function resembling double-opponent cells in the primary visual cortex (V1). In our model we account for four kinds of receptive field surround, i.e. full, far, iso- and orthogonal-orientation, whose contributions are contrast-dependant. The output signal fromV1 is pooled in its perpendicular direction by larger V2 neurons employing a contrast-variant centre-surround kernel. We further introduce a feedback connection from higher-level visual areas to the lower ones. The results of our model on three benchmark datasets show a big improvement compared to the current non-learning and biologically-inspired state-of-the-art algorithms while being competitive to the learning-based methods

Spatial context in the early visual system

Important visual objects in our everyday life, such as fellow people, passing cars or birds perhaps, are not point-like structures but often occupy considerable amounts of the visual field. However, each photoreceptor in our eyes samples just a tiny portion of the visual field and somehow the visual system should integrate these local signals. This process takes place mainly in the visual cortex and, while higher-order visual areas play an important role in perception of extended structures, it is now well established that visual neurons at the first cortical steps of seeing integrate broad spatial context into their responses. The main purpose of this thesis was to provide detailed information concerning the spatial structure of the mechanisms that underlie integration of spatial context in the early visual system. The opening study of this thesis showed that the antagonistic Gaussians structure that has been used for modeling context integration in single visual neurons provides a relatively accurate description of the process also in the human visual system. The first study introduced a novel method for connecting perceptual and neuroimaging measurements and this method was applied in the second study of this thesis. The second study showed that the human visual system integrates spatial context in terms of its visual field size instead of the size of its cortical representation. The third study showed that context is integrated over an unexpectedly large region of the visual field and that spatially distant context may sometimes increase the contrast response of the visual system. The closing study showed that orientation specificity of the integration of spatial context depends on distance both in single neurons in the macaque primary visual cortex and in human perception. The knowledge acquired in this thesis will be generally useful in applications that require understanding of the human visual system.Arkielämän kannalta tärkeät visuaaliset objektit kuten ihmiset, ohikiitävät autot ja kenties kissat, ovat harvoin pistemäisiä, mutta sen sijaan voivat peittää laajankin alueen näkökentästä. Näköaistinsolut prosessoivat kuvainformaatiota erittäin pieneltä näkökentän alueelta ja näköjärjestelmän tulee jollain tavoin yhdistää nämä paikalliset signaalit. Vaikka näköaivokuoren myöhäisten alueiden merkitys spatiaalisesti laajojen objektien havaitsemisessa onkin merkittävä, nykytietämyksen valossa on kiistatonta että myös varhaisten näköaivokuorten hermosolut integroivat spatiaalista kontekstia laajalta näkökentän alueelta. Tässä väitöskirjassa tutkitaan konteksti-integraation taustalla olevien mekanismien spatiaalista rakennetta varhaisessa näköjärjestelmässä. Väitöskirjan ensimmäisessä osatyössä osoitettiin että konteksti-integraatiota yksittäisissä hermosoluissa kuvaavat kahden antagonistisen Gaussilaisen mallit ovat melko hyviä kuvauksia konteksti-integraatiomekanismien spatiaalisesta rakenteesta myös ihmisen näköjärjestelmässä. Ensimmäisessä osatyössä kehitettiin menetelmä joka mahdollistaa havainto- ja aivokuvantamismittausten uudenlaisen yhdistämisen. Tätä menetelmää sovellettiin toisessa osatyössä, jonka päätulos oli konteksti-integraation riippuvuus ärsykkeen koosta näkökentässä sen sijaan että se olisi sidoksissa ärsykkeen edustuksen kokoon aivokuorella. Kolmannessa osatyössä osoitettiin, että kontekstia integroidaan huomattavan laajalta alueelta ja että spatiaalisesti etäinen konteksti saattaa toisinaan vahvistaa näköjärjestelmän kontrastivastetta. Neljäs tutkimus osoitti, että konteksti-integraation valikoivuus orientaatiolle riippuu etäisyydestä niin ihmisen näköhavainnoissa kuin makaki-apinan ensimmäisen näköaivokuoren soluissakin. Tämän väitöskirjan tuloksia voidaan hyödyntää sovelluksissa joissa tarvitaan tietoa ihmisen näköjärjestelmän toiminnasta

Face segregation and recognition by cortical multi-scale line and edge coding

Models of visual perception are based on image representations in cortical area V1 and higher areas which contain many cell layers for feature extraction. Basic simple, complex and end-stopped cells provide input for line, edge and keypoint detection. In this paper we present an improved method for multi-scale line/edge detection based on simple and complex cells. We illustrate the line/edge representation for object reconstruction, and we present models for multi-scale face (object) segregation and recognition that can be embedded into feedforward dorsal and ventral data streams (the “what” and “where” subsystems) with feedback streams from higher areas for obtaining translation, rotation and scale invariance