Neuronal encoding of natural imagery in dragonfly motion pathways

Vision is the primary sense of humans and most other animals. While the act of seeing seems easy, the neuronal architectures that underlie this ability are some of the most complex of the brain. Insects represent an excellent model for investigating how vision operates as they often lead rich visual lives while possessing relatively simple brains. Among insects, aerial predators such as the dragonfly face additional survival tasks. Not only must aerial predators successfully navigate three-dimensional visual environments, they must also be able to identify and track their prey. This task is made even more difficult due to the complexity of visual scenes that contain detail on all scales of magnification, making the job of the predator particularly challenging. Here I investigate the physiology of neurons accessible through tracts in the third neuropil of the optic lobe of the dragonfly. It is at this stage of processing that the first evidence of both wide-field motion and object detection emerges. My research extends the current understanding of two main pathways in the dragonfly visual system, the wide-field motion pathway and target-tracking pathway. While wide-field motion pathways have been studied in numerous insects, until now the dragonfly wide-field motion pathway remains unstudied. Investigation of this pathway has revealed properties, novel among insects, specifically the purely optical adaptation to motion at both high and low velocities through motion adaptation. Here I characterise these newly described neurons and investigate their adaptation properties. The dragonfly target-tracking pathway has been studied extensively, but most research has focussed on classical stimuli such as gratings and small black objects moving on white monitors. Here I extend previous research, which characterised the behaviour of target tracking neurons in cluttered environments, developing a paradigm to allow numerous properties of targets to be changed while still measuring tracking performance. I show that dragonfly neurons interact with clutter through the previously discovered selective attention system, treating cluttered scenes as collections of target-like features. I further show that this system uses the direction and speed of the target and background as one of the key parameters for tracking success. I also elucidate some additional properties of selective attention including the capacity to select for inhibitory targets or weakly salient features in preference to strongly excitatory ones. In collaboration with colleagues, I have also performed some limited modelling to demonstrate that a selective attention model, which includes switching best explains experimental data. Finally, I explore a mathematical model called divisive normalisation which may partially explain how neurons with large receptive fields can be used to re-establish target position information (lost in a position invariant system) through relatively simple integrations of multiple large receptive field neurons. In summary, my thesis provides a broad investigation into several questions about how dragonflies can function in natural environments. More broadly, my thesis addresses general questions about vision and how complicated visual tasks can be solved via clever strategies employed in neuronal systems and their modelled equivalents.Thesis (Ph.D.) -- University of Adelaide, Adelaide Medical School, 201

Robust Models for Optic Flow Coding in Natural Scenes Inspired by Insect Biology

The extraction of accurate self-motion information from the visual world is a difficult problem that has been solved very efficiently by biological organisms utilizing non-linear processing. Previous bio-inspired models for motion detection based on a correlation mechanism have been dogged by issues that arise from their sensitivity to undesired properties of the image, such as contrast, which vary widely between images. Here we present a model with multiple levels of non-linear dynamic adaptive components based directly on the known or suspected responses of neurons within the visual motion pathway of the fly brain. By testing the model under realistic high-dynamic range conditions we show that the addition of these elements makes the motion detection model robust across a large variety of images, velocities and accelerations. Furthermore the performance of the entire system is more than the incremental improvements offered by the individual components, indicating beneficial non-linear interactions between processing stages. The algorithms underlying the model can be implemented in either digital or analog hardware, including neuromorphic analog VLSI, but defy an analytical solution due to their dynamic non-linear operation. The successful application of this algorithm has applications in the development of miniature autonomous systems in defense and civilian roles, including robotics, miniature unmanned aerial vehicles and collision avoidance sensors

A target-detecting visual neuron in the dragonfly locks-on to selectively attended targets

The visual world projects a complex and rapidly changing image onto the retina of many animal species. This presents computational challenges for those animals reliant on visual processing to provide an accurate representation of the world. One such challenge is parsing a visual scene for the most salient targets, such as the selection of prey amidst a swarm. The ability to selectively prioritize processing of some stimuli over others is known as 'selective attention'. We recently identified a dragonfly visual neuron called 'Centrifugal Small Target Motion Detector 1' (CSTMD1) that exhibits selective attention when presented with multiple, equally salient targets. Here we conducted in vivo, electrophysiological recordings from CSTMD1 in wild-caught male dragonflies (Hemicordulia tau), whilst presenting visual stimuli on an LCD monitor. To identify the target selected in any given trial, we uniquely modulated the intensity of the moving targets (frequency-tagging). We found that the frequency information of the selected target is preserved in the neuronal response, whilst the distracter is completely ignored. We also show that the competitive system that underlies selection in this neuron can be biased by the presentation of a preceding target on the same trajectory, even when it is of lower contrast than an abrupt, novel distracter. With this improved method for identifying and biasing target selection in CSTMD1, the dragonfly provides an ideal animal model system to probe the neuronal mechanisms underlying selective attention.Benjamin H. Lancer, Bernard J.E. Evans, Joseph M. Fabian, David C. O’Carroll, and Steven D. Wiederma

Neural correlates of contrast normalization in the Drosophila visual system

The fruit fly Drosophila melanogaster has long become a paramount model organism for research in life sciences. As a result of the fly’s high temporal resolution and its reliable optomotor response - a reflex that helps it compensate for movements of the environment, - Drosophila lends itself exceptionally well to the study of vision and, in particular, the mechanism of motion detection. In the wild, Drosophila is active throughout the day, with especially high levels of activity at dawn and dusk, the periods of time when the visuals of the environment are changing rapidly. The fruit flies can also be found in a variety of habitats, from the expanse of an open field to the inside of a cluttered kitchen. Altogether, Drosophila encounters a variety of visual statistics it must employ to robustly respond to the outside world and succeed in finding food, escaping predators, and carrying out courtship behavior. In my thesis, I focused on the effects of visual contrast, i.e., differences in brightness in the environment, on the fly’s motion vision. I studied the impact of the surround contrast on the filtering properties of the visual interneurons within the motion detection circuit, including the first direction-selective T4 and T5 cells and their main inputs, and how the fly compensates for the changes in contrast to faithfully match the direction and speed of its movement to the external motion under various contrast conditions. Firstly, in Manuscript 1, we established the existence of contrast normalization in the early visual system of Drosophila and demonstrated its suppressive effect on the response amplitude at higher contrasts. We determined where contrast normalization first arises in the optic lobe and identified the main inputs into the T4 and T5 cells that exhibit contrast normalizing properties. We comprehensively characterized the normalization process: namely, it is fast, not dependent on the direction of motion, its effect comes from outside the receptive field of a cell and increases in strength with the size of the visual surround. Additionally, we demonstrated that the normalization relies on neuronal feedback and showed that adding a contrast normalization stage to the existing models of motion detection improves their robustness, matching their performance to the results obtained in behavioral experiments. In Manuscript 2, we further investigated the effects of contrast normalization on the main inputs to T4 and T5 cells, now focusing on its effect on the filtering properties of the cells. We demonstrated that spatially or temporally dynamic surrounds elicit contrast normalization, while static ones do not. We further showed that, in addition to the suppressive effect on the amplitude, contrast normalization speeds up the kinetics of the response and confirmed that this effect is not due to signal saturation and involves a change in the filtering properties of the cell. In summary, we elucidated the role of contrast normalization in the motion detection circuit in the early visual system of Drosophila, comprehensively described the characteristics of the normalization process, and outlined its effects on the filtering properties of the cells. We also emphasized the potential role of shunting inhibition and narrowed down the search for the main candidates in the contrast normalization mechanism, paving the way for future studies to further delve into the contrast normalization circuit and implementation mechanism

Characterization of aminergic neurons controlling behavioral persistence and motivation in Drosophila melanogaster

Deprivation is at odds with survival. To obliterate their condition of hunger animals engage in costly foraging behavior. This conundrum demands unceasing integration of external sensory processing and internal metabolic monitors. Unsurprisingly, such critical behaviors are translated to strong impulses. If unchecked, however, impulsivity can trap animals in unfavorable behavioral states and prevent them from exploiting other valuable opportunities. Categorically, motivational mechanisms have been proposed as the conduit to comply with or decline a response to a strong impulse. Thus, motivation emerges as a critical determinant for observed animal behavioral variability at a given time. Although neuronal circuit diagrams may be deceptively static, neuromodulation can implement behavioral variability in the nervous systems. Bioamines, such as dopamine and norepinephrine, mediate modulatory impact on intrinsic motivational circuits that govern feeding and reward. Across model organisms, however, how animals integrate and update decision-making based on the current motivational and internal states are still poorly understood at the molecular and circuitry levels. Due to its extensive toolbox and amenable miniature nervous systems, Drosophila melanogaster is poised to enrich the current perspective for these concepts. For Drosophila melanogaster, certain odors are salient cues for long distance foraging events. To explore how starved flies make goal-directed decisions, I developed a novel spherical treadmill paradigm. Through the utilization of high-resolution behavioral analyses and tight control of, otherwise highly turbulent, odor delivery, I found that food-deprived flies tracked vinegar persistently even in the repeated absence of a food reward. Combining this behavioral paradigm with immediate neuronal manipulations revealed that this innate persistence recruited circuits that are traditionally linked with learning and memory in an internal state-dependent manner. TH+ cluster dopaminergic neurons, operators of punishment learning, and Dop1R2 signaling enabled this olfactory-driven persistence. Downstream of these dopaminergic neurons, a single mushroom body output neuron, MVP2 was crucial for persistence. MVP2 was necessary and sufficient to integrate hunger state as the underlying motivational drive for food-seeking persistence. Furthermore, I investigated how this strong impulse is counteracted when a fly reaches its goal, nutritious food. A change from odor tracking to food consumption demands the coordination of different sensory systems and motor control subunits. Norepinephrine is implemented in such global switches; such as fight or flight transitions. Using optogenetic manipulation, I demonstrated that the food-seeking drive was suppressed by, an insect norepinephrine analog, octopaminergic input, via VPM4 neurons. Being connected to MVP2 synaptically, which we showed using high-resolution tracing techniques, and a surrogate for feeding at the neuronal level, VPM4 neurons acted as the inhibitory brake on persistent odor tracking to allow feeding related behavior. As a culmination of novel paradigm development, thermo/optogenetic neuronal manipulations and connectomics, this work presents a neuronal microcircuit that recapitulates the alterations of animal behavior faithfully from odor tracking to olfactory suppression during feeding. Specific subsets of dopaminergic and octopaminergic neurons are found to be mediators of motivationally driven events. My findings provide fresh mechanistic insights on how multimodal integration can occur in the brain, how such systems are prone to the internal states, and offers several plausible explanations on how persistence emerges. Finally, this work might serve as a template to better understand the roles and the functional diversity of mammalian aminergic neurons

Characterization of aminergic neurons controlling behavioral persistence and motivation in Drosophila melanogaster

Deprivation is at odds with survival. To obliterate their condition of hunger animals engage in costly foraging behavior. This conundrum demands unceasing integration of external sensory processing and internal metabolic monitors. Unsurprisingly, such critical behaviors are translated to strong impulses. If unchecked, however, impulsivity can trap animals in unfavorable behavioral states and prevent them from exploiting other valuable opportunities. Categorically, motivational mechanisms have been proposed as the conduit to comply with or decline a response to a strong impulse. Thus, motivation emerges as a critical determinant for observed animal behavioral variability at a given time. Although neuronal circuit diagrams may be deceptively static, neuromodulation can implement behavioral variability in the nervous systems. Bioamines, such as dopamine and norepinephrine, mediate modulatory impact on intrinsic motivational circuits that govern feeding and reward. Across model organisms, however, how animals integrate and update decision-making based on the current motivational and internal states are still poorly understood at the molecular and circuitry levels. Due to its extensive toolbox and amenable miniature nervous systems, Drosophila melanogaster is poised to enrich the current perspective for these concepts. For Drosophila melanogaster, certain odors are salient cues for long distance foraging events. To explore how starved flies make goal-directed decisions, I developed a novel spherical treadmill paradigm. Through the utilization of high-resolution behavioral analyses and tight control of, otherwise highly turbulent, odor delivery, I found that food-deprived flies tracked vinegar persistently even in the repeated absence of a food reward. Combining this behavioral paradigm with immediate neuronal manipulations revealed that this innate persistence recruited circuits that are traditionally linked with learning and memory in an internal state-dependent manner. TH+ cluster dopaminergic neurons, operators of punishment learning, and Dop1R2 signaling enabled this olfactory-driven persistence. Downstream of these dopaminergic neurons, a single mushroom body output neuron, MVP2 was crucial for persistence. MVP2 was necessary and sufficient to integrate hunger state as the underlying motivational drive for food-seeking persistence. Furthermore, I investigated how this strong impulse is counteracted when a fly reaches its goal, nutritious food. A change from odor tracking to food consumption demands the coordination of different sensory systems and motor control subunits. Norepinephrine is implemented in such global switches; such as fight or flight transitions. Using optogenetic manipulation, I demonstrated that the food-seeking drive was suppressed by, an insect norepinephrine analog, octopaminergic input, via VPM4 neurons. Being connected to MVP2 synaptically, which we showed using high-resolution tracing techniques, and a surrogate for feeding at the neuronal level, VPM4 neurons acted as the inhibitory brake on persistent odor tracking to allow feeding related behavior. As a culmination of novel paradigm development, thermo/optogenetic neuronal manipulations and connectomics, this work presents a neuronal microcircuit that recapitulates the alterations of animal behavior faithfully from odor tracking to olfactory suppression during feeding. Specific subsets of dopaminergic and octopaminergic neurons are found to be mediators of motivationally driven events. My findings provide fresh mechanistic insights on how multimodal integration can occur in the brain, how such systems are prone to the internal states, and offers several plausible explanations on how persistence emerges. Finally, this work might serve as a template to better understand the roles and the functional diversity of mammalian aminergic neurons

An Insect-Inspired Target Tracking Mechanism for Autonomous Vehicles

Target tracking is a complicated task from an engineering perspective, especially where targets are small and seen against complex natural environments. Due to the high demand for robust target tracking algorithms a great deal of research has focused on this area. However, most engineering solutions developed for this purpose are often unreliable in real world conditions or too computationally expensive to be used in real-time applications. While engineering methods try to solve the problem of target detection and tracking by using high resolution input images, fast processors, with typically computationally expensive methods, a quick glance at nature provides evidence that practical real world solutions for target tracking exist. Many animals track targets for predation, territorial or mating purposes and with millions of years of evolution behind them, it seems reasonable to assume that these solutions are highly efficient. For instance, despite their low resolution compound eyes and tiny brains, many flying insects have evolved superb abilities to track targets in visual clutter even in the presence of other distracting stimuli, such as swarms of prey and conspecifics. The accessibility of the dragonfly for stable electrophysiological recordings makes this insect an ideal and tractable model system for investigating the neuronal correlates for complex tasks such as target pursuit. Studies on dragonflies identified and characterized a set of neurons likely to mediate target detection and pursuit referred to as ‘small target motion detector’ (STMD) neurons. These neurons are selective for tiny targets, are velocity-tuned, contrast-sensitive and respond robustly to targets even against the motion of background. These neurons have shown several high-order properties which can contribute to the dragonfly’s ability to robustly pursue prey with over a 97% success rate. These include the recent electrophysiological observations of response ‘facilitation’ (a slow build-up of response to targets that move on long, continuous trajectories) and ‘selective attention’, a competitive mechanism that selects one target from alternatives. In this thesis, I adopted a bio-inspired approach to develop a solution for the problem of target tracking and pursuit. Directly inspired by recent physiological breakthroughs in understanding the insect brain, I developed a closed-loop target tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. First, I tested this model in virtual world simulations using MATLAB/Simulink. The results of these simulations show robust performance of this insect-inspired model, achieving high prey capture success even within complex background clutter, low contrast and high relative speed of pursued prey. Additionally, these results show that inclusion of facilitation not only substantially improves success for even short-duration pursuits, it also enhances the ability to ‘attend’ to one target in the presence of distracters. This inspect-inspired system has a relatively simple image processing strategy compared to state-of-the-art trackers developed recently for computer vision applications. Traditional machine vision approaches incorporate elaborations to handle challenges and non-idealities in the natural environments such as local flicker and illumination changes, and non-smooth and non-linear target trajectories. Therefore, the question arises as whether this insect inspired tracker can match their performance when given similar challenges? I investigated this question by testing both the efficacy and efficiency of this insect-inspired model in open-loop, using a widely-used set of videos recorded under natural conditions. I directly compared the performance of this model with several state-of-the-art engineering algorithms using the same hardware, software environment and stimuli. This insect-inspired model exhibits robust performance in tracking small moving targets even in very challenging natural scenarios, outperforming the best of the engineered approaches. Furthermore, it operates more efficiently compared to the other approaches, in some cases dramatically so. Computer vision literature traditionally test target tracking algorithms only in open-loop. However, one of the main purposes for developing these algorithms is implementation in real-time robotic applications. Therefore, it is still unclear how these algorithms might perform in closed-loop real-world applications where inclusion of sensors and actuators on a physical robot results in additional latency which can affect the stability of the feedback process. Additionally, studies show that animals interact with the target by changing eye or body movements, which then modulate the visual inputs underlying the detection and selection task (via closed-loop feedback). This active vision system may be a key to exploiting visual information by the simple insect brain for complex tasks such as target tracking. Therefore, I implemented this insect-inspired model along with insect active vision in a robotic platform. I tested this robotic implementation both in indoor and outdoor environments against different challenges which exist in real-world conditions such as vibration, illumination variation, and distracting stimuli. The experimental results show that the robotic implementation is capable of handling these challenges and robustly pursuing a target even in highly challenging scenarios.Thesis (Ph.D.) -- University of Adelaide, School of Mechanical Engineering, 201

Salience invariance with divisive normalization in higher-order insect neurons

We present a biologically inspired model for estimating the position of a moving target that is invariant to the target's contrast. Our model produces a monotonic relationship between position and output activity using a divisive normalization between the 'receptive fields' of two overlapping, wide-field, small-target motion detector (STMD) neurons. These visual neurons found in flying insects, likely underlie the impressive ability to pursue prey within cluttered environments. Individual STMD responses confound the properties of target contrast, size, velocity and position. Inspired by results from STMD recordings we developed a model using a division operation to overcome the inherent positional ambiguities of integrative neurons. We used genetic algorithms to determine the plausibility of such an operation arising and existing over multiple generations. This method allows the lost information to be recovered without needing additional neuronal pathways.Bernard J. E. Evans, David C. O, Carroll, and Steven D. Wiederma

Salience invariance with divisive normalization in higher-order insect neurons

We present a biologically inspired model for estimating the position of a moving target that is invariant to the target's contrast. Our model produces a monotonic relationship between position and output activity using a divisive normalization between the 'receptive fields' of two overlapping, wide-field, small-target motion detector (STMD) neurons. These visual neurons found in flying insects, likely underlie the impressive ability to pursue prey within cluttered environments. Individual STMD responses confound the properties of target contrast, size, velocity and position. Inspired by results from STMD recordings we developed a model using a division operation to overcome the inherent positional ambiguities of integrative neurons. We used genetic algorithms to determine the plausibility of such an operation arising and existing over multiple generations. This method allows the lost information to be recovered without needing additional neuronal pathways