Sparse Coding Predicts Optic Flow Specificities of Zebrafish Pretectal Neurons

Zebrafish pretectal neurons exhibit specificities for large-field optic flow patterns associated with rotatory or translatory body motion. We investigate the hypothesis that these specificities reflect the input statistics of natural optic flow. Realistic motion sequences were generated using computer graphics simulating self-motion in an underwater scene. Local retinal motion was estimated with a motion detector and encoded in four populations of directionally tuned retinal ganglion cells, represented as two signed input variables. This activity was then used as input into one of two learning networks: a sparse coding network (competitive learning) and backpropagation network (supervised learning). Both simulations develop specificities for optic flow which are comparable to those found in a neurophysiological study (Kubo et al. 2014), and relative frequencies of the various neuronal responses are best modeled by the sparse coding approach. We conclude that the optic flow neurons in the zebrafish pretectum do reflect the optic flow statistics. The predicted vectorial receptive fields show typical optic flow fields but also "Gabor" and dipole-shaped patterns that likely reflect difference fields needed for reconstruction by linear superposition.Comment: Published Conference Paper from ICANN 2018, Rhode

A Neural Model of How the Brain Computes Heading from Optic Flow in Realistic Scenes

Animals avoid obstacles and approach goals in novel cluttered environments using visual information, notably optic flow, to compute heading, or direction of travel, with respect to objects in the environment. We present a neural model of how heading is computed that describes interactions among neurons in several visual areas of the primate magnocellular pathway, from retina through V1, MT+, and MSTd. The model produces outputs which are qualitatively and quantitatively similar to human heading estimation data in response to complex natural scenes. The model estimates heading to within 1.5° in random dot or photo-realistically rendered scenes and within 3° in video streams from driving in real-world environments. Simulated rotations of less than 1 degree per second do not affect model performance, but faster simulated rotation rates deteriorate performance, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.National Science Foundation (SBE-0354378, BCS-0235398); Office of Naval Research (N00014-01-1-0624); National-Geospatial Intelligence Agency (NMA201-01-1-2016

A Neural Model of Motion Processing and Visual Navigation by Cortical Area MST

Cells in the dorsal medial superior temporal cortex (MSTd) process optic flow generated by self-motion during visually-guided navigation. A neural model shows how interactions between well-known neural mechanisms (log polar cortical magnification, Gaussian motion-sensitive receptive fields, spatial pooling of motion-sensitive signals, and subtractive extraretinal eye movement signals) lead to emergent properties that quantitatively simulate neurophysiological data about MSTd cell properties and psychophysical data about human navigation. Model cells match MSTd neuron responses to optic flow stimuli placed in different parts of the visual field, including position invariance, tuning curves, preferred spiral directions, direction reversals, average response curves, and preferred locations for stimulus motion centers. The model shows how the preferred motion direction of the most active MSTd cells can explain human judgments of self-motion direction (heading), without using complex heading templates. The model explains when extraretinal eye movement signals are needed for accurate heading perception, and when retinal input is sufficient, and how heading judgments depend on scene layouts and rotation rates.Defense Research Projects Agency (N00014-92-J-4015); Office of Naval Research (N00014-92-J-1309, N00014-95-1-0409, N00014-95-1-0657, N00014-91-J-4100, N0014-94-I-0597); Air Force Office of Scientific Research (F49620-92-J-0334)

Humans combine the optic flow with static depth cues for robust perception of heading

The retinal flow during normal locomotion contains components due to rotation and translation of the observer. The translatory part of the flow-pattern is informative of heading, because it radiates outward from the direction of heading. However, it is not directly accessible from the retinal flow. Nevertheless, humans can perceive their direction of heading from the compound retinal flow without need for extra-retinal signals that indicate the rotation. Two classes of models have been proposed to explain the visual decomposition of the retinal flow into its constituent parts. One type relies on local operations to remove the rotational part of the flow field. The other type explicitly determines the direction and magnitude of the rotation from the global retinal flow, for subsequent removal. According to the former model, nearby points are most reliable for estimating one's heading. In the latter type of model the quality of the heading estimate depends on the accuracy with which the ego-rotation is determined and is therefore most reliable when based on the most distant points. We report that subjects underestimate the eccentricity of heading, relative to the fixated point in the ground plane, when the visible range of the ground plane is reduced. Moreover we find that in perception of heading, humans can tolerate more noise than the optimal observer (in the least squares sense) would do if only using optic flow. The latter finding argues against both schemes because ultimately both classes of model are limited in their noise tolerance to that of the optimal observer, which uses all information available in the optic flow. Apparently humans use more information than is present in the optic flow. Both aspects of human performance are consistent with the use of static depth information in addition to the optic flow to select the most distant points. Processing of the flow of these selected points provides the most reliable estimate of the ego-rotation. Subsequent estimates of the heading direction, obtained from the translatory component of the flow, are robust with respect to noise. In such a scheme heading estimates are subject to systematic errors, similar to those reported, if the most distant points are not much further away than the fixation point, because the ego-rotation is underestimated

Adaptive structure tensors and their applications

The structure tensor, also known as second moment matrix or Förstner interest operator, is a very popular tool in image processing. Its purpose is the estimation of orientation and the local analysis of structure in general. It is based on the integration of data from a local neighborhood. Normally, this neighborhood is defined by a Gaussian window function and the structure tensor is computed by the weighted sum within this window. Some recently proposed methods, however, adapt the computation of the structure tensor to the image data. There are several ways how to do that. This article wants to give an overview of the different approaches, whereas the focus lies on the methods based on robust statistics and nonlinear diffusion. Furthermore, the dataadaptive structure tensors are evaluated in some applications. Here the main focus lies on optic flow estimation, but also texture analysis and corner detection are considered

A parallel implementation of a multisensor feature-based range-estimation method

There are many proposed vision based methods to perform obstacle detection and avoidance for autonomous or semi-autonomous vehicles. All methods, however, will require very high processing rates to achieve real time performance. A system capable of supporting autonomous helicopter navigation will need to extract obstacle information from imagery at rates varying from ten frames per second to thirty or more frames per second depending on the vehicle speed. Such a system will need to sustain billions of operations per second. To reach such high processing rates using current technology, a parallel implementation of the obstacle detection/ranging method is required. This paper describes an efficient and flexible parallel implementation of a multisensor feature-based range-estimation algorithm, targeted for helicopter flight, realized on both a distributed-memory and shared-memory parallel computer

Non-linear carbon dioxide determination using infrared gas sensors and neural networks with Bayesian regularization

Carbon dioxide gas concentration determination using infrared gas sensors combined with Bayesian regularizing neural networks is presented in this work. Infrared sensor with a measuring range of 0~5% was used to measure carbon dioxide gas concentration within the range 0~15000 ppm. Neural networks were employed to fulfill the nonlinear output of the sensor. The Bayesian strategy was used to regularize the training of the back propagation neural network with a Levenberg-Marquardt (LM) algorithm. By Bayesian regularization (BR), the design of the network was adaptively achieved according to the complexity of the application. Levenberg-Marquardt algorithm under Bayesian regularization has better generalization capability, and is more stable than the classical method. The results showed that the Bayesian regulating neural network was a powerful tool for dealing with the infrared gas sensor which has a large non-linear measuring range and provide precise determination of carbon dioxide gas concentration. In this example, the optimal architecture of the network was one neuron in the input and output layer and two neurons in the hidden layer. The network model gave a relationship coefficient of 0.9996 between targets and outputs. The prediction recoveries were within 99.9~100.0%