Biological model of motion integration and segmentation based on form cues

Active vision is an essential part of every biological organism possessing an eye system: posture, eye movements, visual research, ... All require a motion percept to operate. At the basis of active vision lays the ability to calculate movements of objects in the scene, at least on a sufficient level to react correctly. In this report we present a model of motion integration and segmentation in the first visual cortex areas. Specifically we modeled the first two cortex areas involved in motion processing in the primate: V1 and MT. To be able to process motion correctly a visual system also need to deal with form information. We investigate how form cues coming from the ventral pathway can be used by the V1/MT dorsal pathway to solve some perception problems. By using a recurrent dynamical system between the V1 and MT layers we are able to find out psychophysical results such as motion integration and center-surround effects due to the feedback connections, or end-of-line and 2D features detectors thanks to the shunting inhibition. We propose to modulate this system by a form information coming from the ventral stream and are thus able to explain asymmetric center-surround effects as well as motion segmentation and segregation between extrinsic and intrinsic junctions

Motion integration modulated by form information

ISBN : 978-2-9532965-0-1We propose a model of motion integration modulated by form information, inspired by neurobiological data. Our dynamical system models several key features of the motion processing stream in primate visual cortex. Thanks to a multi-layer architecture incorporating both feedforward-feedback and inhibitive lateral connections, our model is able to solve local motion ambiguities. One important feature of our model is to propose an anitropic integration of motion based on the form information. Our model can be implemented efficiently on GPU and we show its properties on classical psychophysical examples. First, a simple read-out allows us to reproduce the dynamics of ocular following for a moving bar stimulus. Second, we show how our models able to discriminate between extrinsic and intrinsic junctions present in the chopstick illusion. Finally, we show some promising results on real videos

Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism

The dynamics of motion integration show striking similarities when observed at neuronal, psychophysical, and oculomotor levels. Based on the inter-relation and complementary insights given by those dynamics, our goal was to test how basic mechanisms of dynamical cortical processing can be incorporated in a dynamical model to solve several aspects of 2D motion integration and segmentation. Our model is inspired by the hierarchical processing stages of the primate visual cortex: we describe the interactions between several layers processing local motion and form information through feedforward, feedback, and inhibitive lateral connections. Also, following perceptual studies concerning contour integration and physiological studies of receptive fields, we postulate that motion estimation takes advantage of another low level cue, which is luminance smoothness along edges or surfaces, in order to gate recurrent motion diffusion. With such a model, we successfully reproduced the temporal dynamics of motion integration on a wide range of simple motion stimuli: line segments, rotating ellipses, plaids, and barber poles. Furthermore, we showed that the proposed computational rule of luminance-gated diffusion of motion information is sufficient to explain a large set of contextual modulations of motion integration and segmentation in more elaborated stimuli such as chopstick illusions, simulated aperture problems, or rotating diamonds. As a whole, in this paper we proposed a new basal luminance-driven motion integration mechanism as an alternative to less parsimonious models, we carefully investigated the dynamics of motion integration, and we established a distinction between simple and complex stimuli according to the kind of information required to solve their ambiguities

Bifurcation analysis applied to a model of motion integration with a multistable stimulus

A computational study into the motion perception dynamics of a multistable psychophysics stimulus is presented. A diagonally drifting grating viewed through a square aperture is can be perceived as moving in the actual grating direction or in line with the aperture edges (horizontally or vertically). The different percepts are the product of interplay between ambiguous contour cues and specific terminator cues. We present a dynamical model of motion integration that performs direction selection for such a stimulus and link the different percepts to coexisting steady-states of the underlying equations. We apply the powerful tools of bifurcation analysis and numerical continuation to study the changes to the model's solution structure under the variation of parameters. Indeed, we apply these tools in a systematic way, taking into account biological and mathematical constraints, in order to fix model parameters. A region of parameter space is identified for which the model reproduces the qualitative behaviour observed in experiments. The temporal dynamics of motion integration are studied within this region; specifically, the effect of varying the stimulus gain is studied, which allows for qualitative predictions to be made

Navisio: Towards an integrated reading aid system for low vision patients

International audienceWe propose the Navisio software as a new integrated system to help low vision patients read complex electronic documents (here, PDF files) with more comfort. Navisio aims at taking into account main psychophysical results on reading performance of visually impaired patients. To do this, we analyze what are the main factors in uencing reading performance, and review some existing reading aid systems, dealing with printed and electronic documents. Then, we show how Navisio allows to extend the capabilities of existing reading systems, focusing on the facilitation to navigate in complex documents, and on the highly customizable display. Navisio performance was evaluated against a standard CCTV magnifier tool, with 26 low vision patients. Two kinds of texts were proposed (simple and complex documents) elaborated from a standardised text database. Results show a clear advantage of Navisio in terms of reading speed and comfort. Navisio is intended to evolve: we discuss how it could be extended to any scanned document, thanks to recent computer vision approaches in document layout analysis. Further challenging perspectives are also mentioned

Bio-inspired motion estimation – From modelling to evaluation, can biology be a source of inspiration?

We propose a bio-inspired approach to motion estimation based on recent neuroscience findings concerning the motion pathway. Our goal is to identify the key biological features in order to reach a good compromise between bio-inspiration and computational efficiency. Here we choose the neural field formalism which provides a sound mathematical framework to describe the model at a macroscopic scale. Within this framework we define the cortical activity as coupled integro-differential equations and we prove the well-posedness of the model. We show how our model performs on some classical computer vision videos, and we compare its behaviour against the visual system on a simple classical video used in psychophysics. Following this idea, we propose a new benchmark to evaluate models against visual system performance. Baseline results are provided for both bio-inspired and computer vision models. Results confirm the good performance of recent computer vision approaches even on such synthetic stimuli, and also show that taking biology into account in models can improve performance. As a whole, this article affords a considerable insight into how biology can bring new ideas in computer vision at different levels: modelling principles, mathematical formalism and evaluation methodology. Perspectives around this work are promising and cover the addition of delays to constrain propagation as well as the extension of our benchmark to better characterise the visual system performance

Towards a bio-inspired evaluation methodology for motion estimation models

Offering proper evaluation methodology is essential to continue progress in modelling neural mechanisms in vision/visual information processing. Currently, evaluation of motion estimation models lacks a proper methodology for comparing their performance against the visual system. Here, we set the basis for such a new benchmark methodology which is based on human visual performance as measured in psychophysics, ocular following and neurobiology. This benchmark will enable comparisons between different kinds of models, but also it will challenge current motion estimation models and better characterize their properties with respect to visual cortex performance. To do so, we propose a database of image sequences taken from neuroscience and psychophysics literature. In this article, we focus on two aspects of motion estimation, which are the dynamics of motion integration and the respective influence between 1D versus 2D cues. Then, since motion models possibly deal with different kinds of motion representations and scale, we define here two general readouts based on a global motion estimation. Such readouts, namely eye movements and perceived motion will serve as a reference to compare simulated and experimental data. We evaluate the performance of several models on this data to establish the current state of the art. Models chosen for comparison have very different properties and internal mechanisms, such as feedforward normalisation of V1 and MT processing and recurrent feedback. As a whole, we provide here the basis for a valuable evaluation methodology to unravel the fundamental mechanisms of the visual cortex in motion perception. Our database is freely available on the web together with scoring instructions and results at http://www-sop.inria.fr/neuromathcomp/software/motionpsychobenchOffrir une méthodologie d'évaluation est essentiel pour la recherche en modélisation des mécanismes neuraux impliqués dans la vision. Actuellement, il manque à l'évaluation des modèles d'estimation du mouvement une méthodologie bien définie permettant de comparer leurs performances avec celles du système visuel. Ici nous posons les bases d'un tel banc d'essai, basé sur les performances visuelles des humains telles que mesurées en psychophysique, en oculo-motricité, et en neurobiologie. Ce banc d'essai permettra de comparer différents modèles, mais aussi de mieux caractériser leurs propriétés en regard du comportement du système visuel. Dans ce but, nous proposons un ensemble de séquences vidéos, issues des expérimentations en neurosciences et en psychophysique. Dans cet article, nous mettons l'accent sur deux principaux aspects de l'estimation du mouvement~: les dynamiques d'intégration du mouvement, et les influences respectives des informations 1D par rapport aux informations 2D. De là, nous définissons deux «~lectures~» basés sur l'estimation du mouvement global. De telles «~lectures~», nommément les mouvements des yeux, et le mouvement perçu, serviront de référence pour comparer les données expérimentales et simulées. Nous évaluons les performances de différents modèles sur ces stimuli afin d'établir un état de l'art des modèles d'intégration du mouvement. Les modèles comparés sont choisis en fonction de leurs grandes différences en terme de propriétes et de mécanismes internes (rétroaction, normalisation). En définitive, nous établissons dans ce travail les bases d'une méthodologie d'évaluation permettant de découvrir les mécanismes fondamentaux du cortex visuel dédié à la perception du mouvement. Notre jeu de stimuli est librement accessible sur Internet, accompagné d'instructions pour l'évaluation, et de résultats, à l'adresse~: http://www-sop.inria.fr/neuromathcomp/software/motionpsychobenc

Navisio: Towards an integrated reading aid system for low vision patients

International audienceWe propose the Navisio software as a new integrated system to help low vision patients read complex electronic documents (here, PDF files) with more comfort. Navisio aims at taking into account main psychophysical results on reading performance of visually impaired patients. To do this, we analyze what are the main factors in uencing reading performance, and review some existing reading aid systems, dealing with printed and electronic documents. Then, we show how Navisio allows to extend the capabilities of existing reading systems, focusing on the facilitation to navigate in complex documents, and on the highly customizable display. Navisio performance was evaluated against a standard CCTV magnifier tool, with 26 low vision patients. Two kinds of texts were proposed (simple and complex documents) elaborated from a standardised text database. Results show a clear advantage of Navisio in terms of reading speed and comfort. Navisio is intended to evolve: we discuss how it could be extended to any scanned document, thanks to recent computer vision approaches in document layout analysis. Further challenging perspectives are also mentioned

Modélisation de la dynamique de l'intégration contextuelle du mouvement chez le primate

This thesis addresses the study of motion integration in the primate. Based on anatomical and functional knowledge of two cortical areas involved in motion perception, namely v1 and mt, we explain various perceptual and oculo-motor responses found in the literature. First, we build a recurrent model of motion integration where a minimal number of cortical interactions are assumed. Pro- posing a simple readout mechanism, we are able to reproduce not only motion perception but also the dynamics of smooth pursuit eye movements on various line figures and gratings viewed through different apertures. Second, following perceptual studies concerning motion integration and physiological studies of re- ceptive fields, we construct another dynamical model where motion information is gated by form cues. To this end, we postulate that the visual cortex takes ad- vantage of luminance smoothness in order to gate motion diffusion. Such an ele- mentary diffusion mechanism allows to solve various contextual problems where extrinsic junctions should be eliminated, without relying on complex junction de- tectors or depth computation. Finally, we rewrite the initial dynamical model into the neural fields formalism in order to mathematically analyse its properties. We incorporate the multiplicative feedback term into the formalism, and prove the existence and uniqueness of the solution. To generalise the comparison against visual performance, we propose a new evaluation methodology based on human visual performance and design a database of image sequences taken from biology and psychophysics literature. Indeed, offering proper evaluation methodology is essential to continue progress in modelling the neural mechanisms involved in motion processing. To conclude, we investigate the performances of our neural fields model by comparison against state of the art computer vision approaches and sequences. We find that, despite its original objective, this model gives results comparable to recent computer vision approaches of motion estimation.Dans cette thèse, nous étudions l'intégration du mouvement chez le primate. En se basant sur les connaissances actuelles concernant l'anatomie et les fonctions de deux aires corticales impliquées dans le mouvement, v1 et mt, nous expliquons un certain nombre de réponses perceptuelles et oculo-motrices rapportées dans la littérature. Tout d'abord, nous construisons un modèle récurrent d'intégration du mouvement se basant sur un nombre minimal d'hypothèses concernant les inter- actions corticales. En proposant un simple mécanisme de " lecture ", nous sommes capable de reproduire non seulement la perception, mais aussi les dynamiques oculaires de poursuite sur des stimuli de type ligne ou grille. De là, en se bas- ant des études psychophysiques sur l'intégration du mouvement et sur des études physiologique concernant les champs récepteurs, nous construisons un deuxième modèle dynamique dans lequel l'information concernant le mouvement est di- rigée par un signal de forme. Pour cela, nous postulons que le cortex visuel utilise la régularité de la luminance pour diriger la diffusion du mouvement. Un tel mécan- isme élémentaire de diffusion permet de résoudre des problèmes contextuels, dans lesquels les jonctions extrinsèques doivent être ignorées, sans avoir besoin d'util- iser des mécanismes plus complexes tels que les détecteurs de jonctions ou le calcul de profondeur. Enfin, nous reformulons le modèle initial dans le cadre du form- alisme des champs neuronaux afin d'analyser mathématiquement ses propriétés. Nous incorporons la rétroaction multiplicative dans le formalisme et prouvons l'existence et l'unicité de la solution. Afin de généraliser les comparaisons aux per- formances du système visuel, nous proposons une nouvelle méthodologie d'évalu- ation basée sur les performances du système visuel humain, accompagnée d'une série de vidéos issues de la littérature biologique et psychophysique. En effet, une méthodologie d'évaluation adaptée nous semble essentielle afin de continuer les progrès en modélisation des mécanismes neuraux impliqués dans le traitement du mouvement. Pour conclure, nous analysons les performances de notre modèle d'intégration du mouvement en l'appliquant à des problèmes classiques et récents issus de la vision par ordinateur. En dépit de son objectif initial, notre modèle est capable de donner des résultats comparables aux récentes approches proposées en vision par ordinateur au niveau de l'estimation du mouvement