Detecting Invasive Insects Using Uncewed Aerial Vehicles and Variational Autoencoders

In this thesis, we use machine learning techniques to address limitations in our ability to monitor pest insect migrations. Invasive insect populations, such as the brown marmorated stink bug (BMSB), cause significant economic and environmental damages. In order to mitigate these damages, tracking BMSB migration is vital, but it also poses a challenge. The current state-of-the-art solution to track insect migrations is called mark-release-recapture. In mark-release-recapture, a researcher marks insects with a fluorescent powder, releases them back into the wild, and searches for the insects using ultra-violet flashlights at suspected migration destination locations. However, this involves a significant amount of labor and has a low recapture rate. By automating the insect search step, the recapture rate can be improved, reducing the amount of labor required in the process and improving the quality of the data. We propose a solution to the BMSB migration tracking problem using an unmanned aerial vehicle (UAV) to collect video data of the area of interest. Our system uses an ultra violet (UV) lighting array and digital cameras mounted on the bottom of the UAV, as well as artificial intelligence algorithms such as convolutional neural networks (CNN), and multiple hypotheses tracking (MHT) techniques. Specifically, we propose a novel computer vision method for insect detection using a Convolutional Variational Auto Encoder (CVAE). Our experimental results show that our system can detect BMSB with high precision and recall, outperforming the current state-of-the-art. Additionally, we associate insect observations using MHT, improving detection results and accurately counting real-world insects

Cluster identification and separation in the growing self-organizing map: Application in protein sequence classification

Growing self-organizing map (GSOM) has been introduced as an improvement to the self-organizing map (SOM) algorithm in clustering and knowledge discovery. Unlike the traditional SOM, GSOM has a dynamic structure which allows nodes to grow reflecting the knowledge discovered from the input data as learning progresses. The spread factor parameter (SF) in GSOM can be utilized to control the spread of the map, thus giving an analyst a flexibility to examine the clusters at different granularities. Although GSOM has been applied in various areas and has been proven effective in knowledge discovery tasks, no comprehensive study has been done on the effect of the spread factor parameter value to the cluster formation and separation. Therefore, the aim of this paper is to investigate the effect of the spread factor value towards cluster separation in the GSOM. We used simple k-means algorithm as a method to identify clusters in the GSOM. By using Davies-Bouldin index, clusters formed by different values of spread factor are obtained and the resulting clusters are analyzed. In this work, we show that clusters can be more separated when the spread factor value is increased. Hierarchical clusters can then be constructed by mapping the GSOM clusters at different spread factor values. © 2009 Springer-Verlag London Limited

Data-driven modeling of the olfactory neural codes and their dynamics in the insect antennal lobe

Recordings from neurons in the insects' olfactory primary processing center, the antennal lobe (AL), reveal that the AL is able to process the input from chemical receptors into distinct neural activity patterns, called olfactory neural codes. These exciting results show the importance of neural codes and their relation to perception. The next challenge is to \emph{model the dynamics} of neural codes. In our study, we perform multichannel recordings from the projection neurons in the AL driven by different odorants. We then derive a neural network from the electrophysiological data. The network consists of lateral-inhibitory neurons and excitatory neurons, and is capable of producing unique olfactory neural codes for the tested odorants. Specifically, we (i) design a projection, an odor space, for the neural recording from the AL, which discriminates between distinct odorants trajectories (ii) characterize scent recognition, i.e., decision-making based on olfactory signals and (iii) infer the wiring of the neural circuit, the connectome of the AL. We show that the constructed model is consistent with biological observations, such as contrast enhancement and robustness to noise. The study answers a key biological question in identifying how lateral inhibitory neurons can be wired to excitatory neurons to permit robust activity patterns

Aerodynamic performance of a free-flying dragonfly—A span-resolved investigation

We present a quantitative characterization of the unsteady aerodynamic features of a live, free-flying dragonfly under a well-established flight condition. In particular, our investigations cover the span-wise features of vortex interactions between the fore- and hind-pairs of wings that could be a distinctive feature of a high aspect ratio tandem flapping wing pair. Flapping kinematics and dynamic wing-shape deformation of a dragonfly were measured by tracking painted landmarks on the wings. Using it as the input, computational fluid dynamics analyses were conducted, complemented with time-resolved particle image velocimetry flow measurements to better understand the aerodynamics associated with a dragonfly. The results show that the flow structures around hindwing’s inner region are influenced by forewing’s leading edge vortex, while those around hindwing’s outer region are more influenced by forewing’s shed trailing edge vortex. Using a span-resolved approach, we found that the forewing–hindwing interactions affect the horizontal force (thrust) generation of the hindwing most prominently and the modulation of the force generation is distributed evenly around the midspan. Compared to operating in isolation, the thrust of the hindwing is largely increased during upstroke, albeit the drag is also slightly increased during the downstroke. The vertical force generation is moderately affected by the forewing–hindwing interactions and the modulation takes place in the outer 40% of the hindwing span during the downstroke and in the inner 60% of the span during the upstroke

3D pose estimation of flying animals in multi-view video datasets

Flying animals such as bats, birds, and moths are actively studied by researchers wanting to better understand these animals’ behavior and flight characteristics. Towards this goal, multi-view videos of flying animals have been recorded both in lab- oratory conditions and natural habitats. The analysis of these videos has shifted over time from manual inspection by scientists to more automated and quantitative approaches based on computer vision algorithms. This thesis describes a study on the largely unexplored problem of 3D pose estimation of flying animals in multi-view video data. This problem has received little attention in the computer vision community where few flying animal datasets exist. Additionally, published solutions from researchers in the natural sciences have not taken full advantage of advancements in computer vision research. This thesis addresses this gap by proposing three different approaches for 3D pose estimation of flying animals in multi-view video datasets, which evolve from successful pose estimation paradigms used in computer vision. The first approach models the appearance of a flying animal with a synthetic 3D graphics model and then uses a Markov Random Field to model 3D pose estimation over time as a single optimization problem. The second approach builds on the success of Pictorial Structures models and further improves them for the case where only a sparse set of landmarks are annotated in training data. The proposed approach first discovers parts from regions of the training images that are not annotated. The discovered parts are then used to generate more accurate appearance likelihood terms which in turn produce more accurate landmark localizations. The third approach takes advantage of the success of deep learning models and adapts existing deep architectures to perform landmark localization. Both the second and third approaches perform 3D pose estimation by first obtaining accurate localization of key landmarks in individual views, and then using calibrated cameras and camera geometry to reconstruct the 3D position of key landmarks. This thesis shows that the proposed algorithms generate first-of-a-kind and leading results on real world datasets of bats and moths, respectively. Furthermore, a variety of resources are made freely available to the public to further strengthen the connection between research communities

Asset Protection in Escorting using Multi-Robot Systems

Swarm robotics is a field dedicated to the study of the design and development of certain multi-robot systems. Often times, these groups prove to be more beneficial than a single complex robot as swarms typically provide a more robust and potentially more efficient solution. One such case is the task of escorting a specified target while addressing any potential threats discovered in the environment. In this work, a control algorithm for a high volume, decentralized, homogeneous robot swarm was developed based upon a technique commonly used to model incompressible fluids known as Smoothed Particle Hydrodynamics (SPH). This proposed solution to the asset protection problem was tested against a more commonly accepted method for robot navigation known as potential fields. An alternate algorithm was developed based on this technique and manipulated to perform the same basic duty of asset protection. Both algorithms were tested in simulation using ARGoS as an environment and Swarmanoid’s Footbots as robot models. Five experiments were run in order to examine the functionality of both of these algorithms in relation to formation control and the protection of a mobile asset from mobile threats. The results proved the proposed SPH based algorithm comparable to the potential fields based method while minimizing the escape window and having a slightly higher response rate to introduced threats. These results hint that the concept of using fluid models for control of high volume swarms should further be explored and seriously considered as a potential solution to the asset protection problem

Taking a goal-centred dynamic snapshot as a possibility for local homing in initially naïve bumblebees

Lobecke A, Kern R, Egelhaaf M. Taking a goal-centred dynamic snapshot as a possibility for local homing in initially naïve bumblebees. The Journal of Experimental Biology. 2018;221(2): jeb168674.It is essential for central place foragers, such as bumblebees, to return reliably to their nest. Bumblebees, leaving their inconspicuous nest hole for the first time need to gather and learn sufficient information about their surroundings to allow them to return to their nest at the end of their trip, instead of just flying away to forage. Therefore, we assume an intrinsic learning programme that manifests itself in the flight structure immediately after leaving the nest for the first time. In this study, we recorded and analysed the first outbound flight of individually marked naïve bumblebees in an indoor environment. We found characteristic loop-like features in the flight pattern that appear to be necessary for the bees to acquire environmental information and might be relevant for finding the nest hole after a foraging trip. Despite common features in their spatio- temporal organisation, first departure flights from the nest are characterised by a high level of variability in their loop-like flight structure across animals. Changes in turn direction of body orientation, for example, are distributed evenly across the entire area used for the flights without anysystematic relationship to the nest location. By considering the common flight motifs and this variability, we came to the hypothesis that a kind of dynamic snapshot is taken during the early phase of departure flights centred at the nest location. The quality of this snapshot is hypothesised to be ‘tested’ during the later phases of the departure flights concerning its usefulness for local homing

Impact across ecosystem boundaries-Does Bti application change quality and composition of the diet of riparian spiders?

Emerging aquatic insects link aquatic and adjacent terrestrial food webs by subsidizing terrestrial predators with high -quality prey. One of the main constituents of aquatic subsidy, the non-biting midges (Chironomidae), showed altered emergence dynamics in response to the mosquito control agent Bacillus thuringiensis var. israelensis (Bti). As riparian spi-ders depend on aquatic subsidy, they may be affected by such changes in prey availability. Thus, we conducted a field study in twelve floodplain pond mesocosms (FPMs), six were treated with Bti (2.88 x 109 ITU/ha, VectoBac WDG) three times, to investigate if the Bti-induced shift in chironomid emergence dynamics is reflected in their nutritional value and in the diet of riparian spiders. We measured the content of proteins, lipids, glycogen, and carbohydrates in emerged Chironomidae, and determined the stable isotope ratios of female Tetragnatha extensa, a web-building spi-der living in the riparian vegetation of the FPMs. We analysed the proportion of aquatic prey in spiders' diet, niche size, and trophic position. While the content of nutrients and thus the prey quality was not significantly altered by Bti, ef-fects on the spiders' diet were observed. The trophic position of T. extensa from Bti-treated FPMs was lower compared to the control while the aquatic proportion was only minimally reduced. We assume that spiders fed more on terrestrial prey but also on other aquatic organisms such as Baetidae, whose emergence was unaffected by Bti. In contrast to the partly predaceous Chironomidae, consumption of aquatic and terrestrial primary consumers potentially explains the observed lower trophic position of spiders from Bti-treated FPMs. As prey organisms vary in their quality the suggested dietary shift could transfer previously observed effects of Bti to riparian spiders conceivably affecting their populations. Our results further support that anthropogenic stressors in aquatic ecosystems may translate to terrestrial predators through aquatic subsidy