Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modelling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research of insects' visual systems in the literature. These motion perception models or neural networks comprise the looming sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation sensitive neural systems of direction selective neurons (DSNs) in fruit flies, bees and locusts, as well as the small target motion detectors (STMDs) in dragonflies and hover flies. We also review the applications of these models to robots and vehicles. Through these modelling studies, we summarise the methodologies that generate different direction and size selectivity in motion perception. At last, we discuss about multiple systems integration and hardware realisation of these bio-inspired motion perception models

A neurobiological and computational analysis of target discrimination in visual clutter by the insect visual system.

Some insects have the capability to detect and track small moving objects, often against cluttered moving backgrounds. Determining how this task is performed is an intriguing challenge, both from a physiological and computational perspective. Previous research has characterized higher-order neurons within the fly brain known as 'small target motion detectors‘ (STMD) that respond selectively to targets, even within complex moving surrounds. Interestingly, these cells still respond robustly when the velocity of the target is matched to the velocity of the background (i.e. with no relative motion cues). We performed intracellular recordings from intermediate-order neurons in the fly visual system (the medulla). These full-wave rectifying, transient cells (RTC) reveal independent adaptation to luminance changes of opposite signs (suggesting separate 'on‘ and 'off‘ channels) and fast adaptive temporal mechanisms (as seen in some previously described cell types). We show, via electrophysiological experiments, that the RTC is temporally responsive to rapidly changing stimuli and is well suited to serving an important function in a proposed target-detecting pathway. To model this target discrimination, we use high dynamic range (HDR) natural images to represent 'real-world‘ luminance values that serve as inputs to a biomimetic representation of photoreceptor processing. Adaptive spatiotemporal high-pass filtering (1st-order interneurons) shapes the transient 'edge-like‘ responses, useful for feature discrimination. Following this, a model for the RTC implements a nonlinear facilitation between the rapidly adapting, and independent polarity contrast channels, each with centre-surround antagonism. The recombination of the channels results in increased discrimination of small targets, of approximately the size of a single pixel, without the need for relative motion cues. This method of feature discrimination contrasts with traditional target and background motion-field computations. We show that our RTC-based target detection model is well matched to properties described for the higher-order STMD neurons, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter‘ to successfully detect many targets from the background. The model produces robust target discrimination across a biologically plausible range of target sizes and a range of velocities. We show that the model for small target motion detection is highly correlated to the velocity of the stimulus but not other background statistics, such as local brightness or local contrast, which normally influence target detection tasks. From an engineering perspective, we examine model elaborations for improved target discrimination via inhibitory interactions from correlation-type motion detectors, using a form of antagonism between our feature correlator and the more typical motion correlator. We also observe that a changing optimal threshold is highly correlated to the value of observer ego-motion. We present an elaborated target detection model that allows for implementation of a static optimal threshold, by scaling the target discrimination mechanism with a model-derived velocity estimation of ego-motion. Finally, we investigate the physiological relevance of this target discrimination model. We show that via very subtle image manipulation of the visual stimulus, our model accurately predicts dramatic changes in observed electrophysiological responses from STMD neurons.Thesis (Ph.D.) - University of Adelaide, School of Molecular and Biomedical Science, 200

Modality-specific circuits for skylight orientation in the fly visual system

The fly eye contains different subtypes of unit eyes (ommatidia) with molecularly and morphologically specialized photoreceptors for comparing either between different wavelengths (color vision) or between different angles of the linearly polarized skylight (polarization vision). However, microcircuit differences between those parts of the columnar medulla neuropil computing color versus polarization remain largely unknown. There is virtually nothing known about the circuit elements immediately downstream of polarization-sensitive photoreceptors in the ‘dorsal rim area’ (DRA). In this work, I described the cellular and synaptic architecture of medulla columns that receive skylight polarization input from DRA photoreceptors. I showed that only in the DRA region, R7 and R8 photoreceptors resemble each other by targeting their axons to the same medulla layer. However, within this layer DRA R7 and R8 connect to morphologically distinct Dm target cells (called Dm-DRA1 and Dm-DRA2, respectively). Both Dm-DRA cell types are modality-specific by avoiding contact with color-sensitive photoreceptors. Using the genetic toolbox of Drosophila such as activity-dependent GFP-reconstitution across synaptic partners (GRASP) and the genetically inducible trans-synaptic tracer ‘trans-Tango’, I confirmed that Dm-DRA1 and Dm-DRA2 are the specific post-synaptic targets of DRA.R7 or DRA.R8, respectively. Neither Dm-DRAs overlap with the main synaptic targets of color-sensitive R7 cells (called Dm8 cells), revealing for the first time that skylight polarization is processed by separate modality-specific circuits in the early visual system. These modality-specific differences are not limited only Dm-DRA cells. I described modality-specific cellular and synaptic specializations in other optic lobe cell types in the DRA region of the medulla: the dendritic arbors of certain cell types (neuromodulatory cells and visual projection neurons) specifically avoid the DRA region. Furthermore, Transmedullary (Tm) cells that are post-synaptic to color-sensitive photoreceptors showed modality-specific differences in connectivity or were absent from the DRA. Finally, I contributed a study describing the cellular organization of the ‘anterior visual pathway’ that carries skylight information from the eye to the central brain. In this study, I showed that an optic glomerulus called the anterior optic tubercle (AOTU) receives direct information via different classes of medulla-to-tubercle (MeTu) neurons, terminating in different subdomains of the AOTU. Finally, we hypothesize that different classes of MeTu cells carry different types of skylight information to the central brain via parallel pathways

Review

Detecting the direction of image motion is a fundamental component of visual computation, essential for survival of the animal. However, at the level of individual photoreceptors, the direction in which the image is shifting is not explicitly represented. Rather, directional motion information needs to be extracted from the photoreceptor array by comparing the signals of neighboring units over time. The exact nature of this process as implemented in the visual system of the fruit fly Drosophila melanogaster has been studied in great detail, and much progress has recently been made in determining the neural circuits giving rise to directional motion information. The results reveal the following: (1) motion information is computed in parallel ON and OFF pathways. (2) Within each pathway, T4 (ON) and T5 (OFF) cells are the first neurons to represent the direction of motion. Four subtypes of T4 and T5 cells exist, each sensitive to one of the four cardinal directions. (3) The core process of direction selectivity as implemented on the dendrites of T4 and T5 cells comprises both an enhancement of signals for motion along their preferred direction as well as a suppression of signals for motion along the opposite direction. This combined strategy ensures a high degree of direction selectivity right at the first stage where the direction of motion is computed. (4) At the subsequent processing stage, tangential cells spatially integrate direct excitation from ON and OFF-selective T4 and T5 cells and indirect inhibition from bi-stratified LPi cells activated by neighboring T4/T5 terminals, thus generating flow-field-selective responses

Functional imaging of the neural components of Drosophila motion detection

In order to safely move through the environment, visually-guided animals use several types of visual cues for orientation. Optic flow provides faithful information about ego-motion and can thus be used to maintain a straight course. Additionally, local motion cues or landmarks indicate potentially interesting targets or signal danger, triggering approach or avoidance, respectively. The visual system must reliably and quickly evaluate these cues and integrate this information in order to orchestrate behavior. The underlying neuronal computations for this remain largely inaccessible in higher organisms, such as in humans, but can be studied experimentally in more simple model species. The fly Drosophila, for example, relies heavily on such visual cues during its impressive flight maneuvers. Additionally, it is genetically and physiologically accessible. Therefore, it is regarded as an ideal model organism for exploring neuronal computations underlying visual processing. During my PhD-thesis, I characterized neurons presynaptic to direction selective lobula plate tangential cells by exploiting the genetic toolbox of the fruit fly in combination with in-vivo imaging. The use of genetically encoded calcium indicators and two-photon microscopy allowed me to directly investigate response properties of small columnar neurons upstream of lobula plate wide field neurons. In the highly collaborative environment of our lab my imaging experiments were complemented by several other approaches, including electrophysiological and behavioral experiments, along with modeling which resulted in the publications that comprise this cumulative dissertation. Measuring calcium signals in T4 and T5 cells in the first study, established that both populations of neurons exhibit direction selective response properties. Furthermore, T4 cells only respond to moving bright edges, whereas T5 cells encode exclusively dark edge motion. Silencing the synaptic output of T4 and T5 separately, we were able to determine that both lobula plate tangential cell responses as well as the turning behavior of walking flies were impaired only to bright or dark edges, respectively. We thus proposed that the detection of the direction of visual motion must happen either presynaptic to, or on the dendrites of T4 and T5 neurons, and that this computation takes place independently for brightness increments and decrements. The second paper published in 2014 was motivated by an anatomical study that found an asymmetric wiring between L2 and L4 cells with the dendrites of Tm2 in the distal medulla. Using two-photon calcium imaging and neuronal silencing combined with postsynaptic electrophysiological recordings, we probed the contribution of L4 and Tm2 in the OFF pathway of Drosophila motion vision. We found that while Tm2 have small, isotropic, laterally inhibited receptive fields, L4 cells respond to both, small and large field darkening. Blocking the output of both cell types resulted in a strong impairment of OFF motion vision. In contrast to the anatomical prediction, we did not observe any directional effects for either of the cells

Dynamic Signal Compression for Robust Motion Vision in Flies

Sensory systems need to reliably extract information from highly variable natural signals. Flies, for instance, use optic flow to guide their course and are remarkably adept at estimating image velocity regardless of image statistics. Current circuit models, however, cannot account for this robustness. Here, we demonstrate that the Drosophila visual system reduces input variability by rapidly adjusting its sensitivity to local contrast conditions. We exhaustively map functional properties of neurons in the motion detection circuit and find that local responses are compressed by surround contrast. The compressive signal is fast, integrates spatially, and derives from neural feedback. Training convolutional neural networks on estimating the velocity of natural stimuli shows that this dynamic signal compression can close the performance gap between model and organism. Overall, our work represents a comprehensive mechanistic account of how neural systems attain the robustness to carry out survival-critical tasks in challenging real-world environments

The role of direction-selective visual interneurons T4 and T5 in Drosophila orientation behavior

In order to safely move through the environment, visually-guided animals use several types of visual cues for orientation. Optic flow provides faithful information about ego-motion and can thus be used to maintain a straight course. Additionally, local motion cues or landmarks indicate potentially interesting targets or signal danger, triggering approach or avoidance, respectively. The visual system must reliably and quickly evaluate these cues and integrate this information in order to orchestrate behavior. The underlying neuronal computations for this remain largely inaccessible in higher organisms, such as in humans, but can be studied experimentally in more simple model species. The fly Drosophila, for example, heavily relies on such visual cues during its impressive flight maneuvers. Additionally, it is genetically and physiologically accessible. Hence, it can be regarded as an ideal model organism for exploring neuronal computations during visual processing. In my PhD studies, I have designed and built several autonomous virtual reality setups to precisely measure visual behavior of walking flies. The setups run in open-loop and in closed-loop configuration. In an open-loop experiment, the visual stimulus is clearly defined and does not depend on the behavioral response. Hence, it allows mapping of how specific features of simple visual stimuli are translated into behavioral output, which can guide the creation of computational models of visual processing. In closedloop experiments, the behavioral response is fed back onto the visual stimulus, which permits characterization of the behavior under more realistic conditions and, thus, allows for testing of the predictive power of the computational models. In addition, Drosophila’s genetic toolbox provides various strategies for targeting and silencing specific neuron types, which helps identify which cells are needed for a specific behavior. We have focused on visual interneuron types T4 and T5 and assessed their role in visual orientation behavior. These neurons build up a retinotopic array and cover the whole visual field of the fly. They constitute major output elements from the medulla and have long been speculated to be involved in motion processing. This cumulative thesis consists of three published studies: In the first study, we silenced both T4 and T5 neurons together and found that such flies were completely blind to any kind of motion. In particular, these flies could not perform an optomotor response anymore, which means that they lost their normally innate following responses to motion of large-field moving patterns. This was an important finding as it ruled out the contribution of another system for motion vision-based behaviors. However, these flies were still able to fixate a black bar. We could show that this behavior is mediated by a T4/T5-independent flicker detection circuitry which exists in parallel to the motion system. In the second study, T4 and T5 neurons were characterized via twophoton imaging, revealing that these cells are directionally selective and have very similar temporal and orientation tuning properties to directionselective neurons in the lobula plate. T4 and T5 cells responded in a contrast polarity-specific manner: T4 neurons responded selectively to ON edge motion while T5 neurons responded only to OFF edge motion. When we blocked T4 neurons, behavioral responses to moving ON edges were more impaired than those to moving OFF edges and the opposite was true for the T5 block. Hence, these findings confirmed that the contrast polarityspecific visual motion pathways, which start at the level of L1 (ON) and L2 (OFF), are maintained within the medulla and that motion information is computed twice independently within each of these pathways. Finally, in the third study, we used the virtual reality setups to probe the performance of an artificial microcircuit. The system was equipped with a camera and spherical fisheye lens. Images were processed by an array of Reichardt detectors whose outputs were integrated in a similar way to what is found in the lobula plate of flies. We provided the system with several rotating natural environments and found that the fly-inspired artificial system could accurately predict the axes of rotation