Texture dependence of motion sensing and free flight behavior in blowflies

Lindemann JP, Egelhaaf M. Texture dependence of motion sensing and free flight behavior in blowflies. Frontiers in Behavioral Neuroscience. 2013;6:92.Many flying insects exhibit an active flight and gaze strategy: purely translational flight segments alternate with quick turns called saccades. To generate such a saccadic flight pattern, the animals decide the timing, direction, and amplitude of the next saccade during the previous translatory intersaccadic interval. The information underlying these decisions is assumed to be extracted from the retinal image displacements (optic flow), which scale with the distance to objects during the intersaccadic flight phases. In an earlier study we proposed a saccade-generation mechanism based on the responses of large-field motion-sensitive neurons. In closed-loop simulations we achieved collision avoidance behavior in a limited set of environments but observed collisions in others. Here we show by open-loop simulations that the cause of this observation is the known texture-dependence of elementary motion detection in flies, reflected also in the responses of large-field neurons as used in our model. We verified by electrophysiological experiments that this result is not an artifact of the sensory model. Already subtle changes in the texture may lead to qualitative differences in the responses of both our model cells and their biological counterparts in the fly's brain. Nonetheless, free flight behavior of blowflies is only moderately affected by such texture changes. This divergent texture dependence of motion-sensitive neurons and behavioral performance suggests either mechanisms that compensate for the texture dependence of the visual motion pathway at the level of the circuits generating the saccadic turn decisions or the involvement of a hypothetical parallel pathway in saccadic control that provides the information for collision avoidance independent of the textural properties of the environment

Wing and body motion during flight initiation in Drosophila revealed by automated visual tracking

The fruit fly Drosophila melanogaster is a widely used model organism in studies of genetics, developmental biology and biomechanics. One limitation for exploiting Drosophila as a model system for behavioral neurobiology is that measuring body kinematics during behavior is labor intensive and subjective. In order to quantify flight kinematics during different types of maneuvers, we have developed a visual tracking system that estimates the posture of the fly from multiple calibrated cameras. An accurate geometric fly model is designed using unit quaternions to capture complex body and wing rotations, which are automatically fitted to the images in each time frame. Our approach works across a range of flight behaviors, while also being robust to common environmental clutter. The tracking system is used in this paper to compare wing and body motion during both voluntary and escape take-offs. Using our automated algorithms, we are able to measure stroke amplitude, geometric angle of attack and other parameters important to a mechanistic understanding of flapping flight. When compared with manual tracking methods, the algorithm estimates body position within 4.4±1.3% of the body length, while body orientation is measured within 6.5±1.9 deg. (roll), 3.2±1.3 deg. (pitch) and 3.4±1.6 deg. (yaw) on average across six videos. Similarly, stroke amplitude and deviation are estimated within 3.3 deg. and 2.1 deg., while angle of attack is typically measured within 8.8 deg. comparing against a human digitizer. Using our automated tracker, we analyzed a total of eight voluntary and two escape take-offs. These sequences show that Drosophila melanogaster do not utilize clap and fling during take-off and are able to modify their wing kinematics from one wingstroke to the next. Our approach should enable biomechanists and ethologists to process much larger datasets than possible at present and, therefore, accelerate insight into the mechanisms of free-flight maneuvers of flying insects

Deleterious effect of suboptimal diet on rest-activity cycle in Anastrepha ludens manifests itself with age.

Activity patterns and sleep-wake cycles are among the physiological processes that change most prominently as animals age, and are often good indicators of healthspan. In this study, we used the video-based high-resolution behavioral monitoring system (BMS) to monitor the daily activity cycle of tephritid fruit flies Anastrepha ludens over their lifetime. Surprisingly, there was no dramatic change in activity profile with respect to age if flies were consistently fed with a nutritionally balanced diet. However, if flies were fed with sugar-only diet, their activity profile decreased in amplitude at old age, suggesting that suboptimal diet affected activity patterns, and its detrimental effect may not manifest itself until the animal ages. Moreover, by simulating different modes of behavior monitoring with a range of resolution and comparing the resulting conclusions, we confirmed the superior performance of video-based monitoring using high-resolution BMS in accurately representing activity patterns in an insect model

Learning recurrent representations for hierarchical behavior modeling

We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network. The network has a discriminative part (classifying actions) and a generative part (predicting motion), whose recurrent cells are laterally connected, allowing higher levels of the network to represent high level phenomena. We test our framework on two types of data, fruit fly behavior and online handwriting. Our results show that 1) taking advantage of unlabeled sequences, by predicting future motion, significantly improves action detection performance when training labels are scarce, 2) the network learns to represent high level phenomena such as writer identity and fly gender, without supervision, and 3) simulated motion trajectories, generated by treating motion prediction as input to the network, look realistic and may be used to qualitatively evaluate whether the model has learnt generative control rules

History dependence in insect flight decisions during odor tracking

Natural decision-making often involves extended decision sequences in response to variable stimuli with complex structure. As an example, many animals follow odor plumes to locate food sources or mates, but turbulence breaks up the advected odor signal into intermittent filaments and puffs. This scenario provides an opportunity to ask how animals use sparse, instantaneous, and stochastic signal encounters to generate goal-oriented behavioral sequences. Here we examined the trajectories of flying fruit flies (Drosophila melanogaster) and mosquitoes (Aedes aegypti) navigating in controlled plumes of attractive odorants. While it is known that mean odor-triggered flight responses are dominated by upwind turns, individual responses are highly variable. We asked whether deviations from mean responses depended on specific features of odor encounters, and found that odor-triggered turns were slightly but significantly modulated by two features of odor encounters. First, encounters with higher concentrations triggered stronger upwind turns. Second, encounters occurring later in a sequence triggered weaker upwind turns. To contextualize the latter history dependence theoretically, we examined trajectories simulated from three normative tracking strategies. We found that neither a purely reactive strategy nor a strategy in which the tracker learned the plume centerline over time captured the observed history dependence. In contrast, “infotaxis”, in which flight decisions maximized expected information gain about source location, exhibited a history dependence aligned in sign with the data, though much larger in magnitude. These findings suggest that while true plume tracking is dominated by a reactive odor response it might also involve a history-dependent modulation of responses consistent with the accumulation of information about a source over multi-encounter timescales. This suggests that short-term memory processes modulating decision sequences may play a role in natural plume tracking

Discovering Blind Spots in Reinforcement Learning

Agents trained in simulation may make errors in the real world due to mismatches between training and execution environments. These mistakes can be dangerous and difficult to discover because the agent cannot predict them a priori. We propose using oracle feedback to learn a predictive model of these blind spots to reduce costly errors in real-world applications. We focus on blind spots in reinforcement learning (RL) that occur due to incomplete state representation: The agent does not have the appropriate features to represent the true state of the world and thus cannot distinguish among numerous states. We formalize the problem of discovering blind spots in RL as a noisy supervised learning problem with class imbalance. We learn models to predict blind spots in unseen regions of the state space by combining techniques for label aggregation, calibration, and supervised learning. The models take into consideration noise emerging from different forms of oracle feedback, including demonstrations and corrections. We evaluate our approach on two domains and show that it achieves higher predictive performance than baseline methods, and that the learned model can be used to selectively query an oracle at execution time to prevent errors. We also empirically analyze the biases of various feedback types and how they influence the discovery of blind spots.Comment: To appear at AAMAS 201

Simulating Flying Insects Using Dynamics and Data-Driven Noise Modeling to Generate Diverse Collective Behaviors

We present a biologically plausible dynamics model to simulate swarms of flying insects. Our formulation, which is based on biological conclusions and experimental observations, is designed to simulate large insect swarms of varying densities. We use a force-based model that captures different interactions between the insects and the environment and computes collision-free trajectories for each individual insect. Furthermore, we model the noise as a constructive force at the collective level and present a technique to generate noise-induced insect movements in a large swarm that are similar to those observed in real-world trajectories. We use a data-driven formulation that is based on pre-recorded insect trajectories. We also present a novel evaluation metric and a statistical validation approach that takes into account various characteristics of insect motions. In practice, the combination of Curl noise function with our dynamics model is used to generate realistic swarm simulations and emergent behaviors. We highlight its performance for simulating large flying swarms of midges, fruit fly, locusts and moths and demonstrate many collective behaviors, including aggregation, migration, phase transition, and escape responses

Biologically Inspired Feedback Design for Drosophila Flight

We use a biologically motivated model of the Drosophila's flight mechanics and sensor processing to design a feedback control scheme to regulate forward flight. The model used for insect flight is the grand unified fly (GUF) [3] simulation consisting of rigid body kinematics, aerodynamic forces and moments, sensory systems, and a 3D environment model. We seek to design a control algorithm that will convert the sensory signals into proper wing beat commands to regulate forward flight. Modulating the wing beat frequency and mean stroke angle produces changes in the flight envelope. The sensory signals consist of estimates of rotational velocity from the haltere organs and translational velocity estimates from visual elementary motion detectors (EMD's) and matched retinal velocity filters. The controller is designed based on a longitudinal model of the flight dynamics. Feedforward commands are generated based on a desired forward velocity. The dynamics are linearized around this operating point and a feedback controller designed to correct deviations from the operating point. The control algorithm is implemented in the GUF simulator and achieves the desired tracking of the forward reference velocities and exhibits biologically realistic responses

Active Vision Strategies in Predation

Visual predation requires precise and accurate behaviour, for which many predators have evolved excellent visual skills. However, an animal's visual abilities are greatly affected by how it moves its eyes, known as active vision. Insects have immobile eyes but can direct their gaze by moving their heads and bodies. This thesis examines three predatory insects with different predatory strategies, to understand the extent to which active vision can be used in predation. The first experimental chapter considers the African praying mantid, Sphodromantis lineola. Praying mantids are stationary terrestrial predators, which use their extremely mobile necks to visually track prey until it is within reach. By using statistical models, we identified what factors elicited strikes and, importantly, their success rate. The timing of head movements greatly increased the chances of strike success, with earlier movements increasing the success rate. The second experimental chapter addresses how darting robber flies, Psilonyx annulatus, aerially attack static prey. Prior to attacking, darting robber flies translate their body around a central point, assessing their prey. After assessment, they attack from a position correlated with the target's absolute size, not its angular size. Prey is beyond the robber fly's stereopsis range during the period of assessment. Assessments of differently sized targets have similarities with the behaviour exhibited by jumping insects, which use motion parallax, a form of active vision, to assess jump distance, suggesting darting robber flies also use motion parallax to predate. The final experimental chapter considers killer flies, Coenosia attenuata, which chase moving targets aerially. Killer flies use a combination of gravity and wing acceleration to increase their speed when chasing prey from above. This increased speed restricts the flies' ability to steer. However, killer flies create strong looming stimuli which may trigger their prey to produce evasive manoeuvres, thereby slowing down. Moreover, by travelling faster towards their prey, killer flies may avoid losing track of it, a real danger when chasing moving prey with low- resolution eyes potentially avoided thanks to active vision. By employing active vision, each of the predators considered can achieve impressive performances, despite relying on very different strategies to capture prey. The use of active vision can increase the success of already excellent visual predators and improve the performance of predator with limited vision. However, active vision can also substantially alter predatory behaviour, leading to a trade- off between the advantages in visual perception active vision can bring and the disadvantage in behavioural efficiency of active vision strategies