High-Resolution Remotely Sensed Small Target Detection by Imitating Fly Visual Perception Mechanism

The difficulty and limitation of small target detection methods for high-resolution remote sensing data have been a recent research hot spot. Inspired by the information capture and processing theory of fly visual system, this paper endeavors to construct a characterized model of information perception and make use of the advantages of fast and accurate small target detection under complex varied nature environment. The proposed model forms a theoretical basis of small target detection for high-resolution remote sensing data. After the comparison of prevailing simulation mechanism behind fly visual systems, we propose a fly-imitated visual system method of information processing for high-resolution remote sensing data. A small target detector and corresponding detection algorithm are designed by simulating the mechanism of information acquisition, compression, and fusion of fly visual system and the function of pool cell and the character of nonlinear self-adaption. Experiments verify the feasibility and rationality of the proposed small target detection model and fly-imitated visual perception method

A neurobiological and computational analysis of target discrimination in visual clutter by the insect visual system.

Some insects have the capability to detect and track small moving objects, often against cluttered moving backgrounds. Determining how this task is performed is an intriguing challenge, both from a physiological and computational perspective. Previous research has characterized higher-order neurons within the fly brain known as 'small target motion detectors‘ (STMD) that respond selectively to targets, even within complex moving surrounds. Interestingly, these cells still respond robustly when the velocity of the target is matched to the velocity of the background (i.e. with no relative motion cues). We performed intracellular recordings from intermediate-order neurons in the fly visual system (the medulla). These full-wave rectifying, transient cells (RTC) reveal independent adaptation to luminance changes of opposite signs (suggesting separate 'on‘ and 'off‘ channels) and fast adaptive temporal mechanisms (as seen in some previously described cell types). We show, via electrophysiological experiments, that the RTC is temporally responsive to rapidly changing stimuli and is well suited to serving an important function in a proposed target-detecting pathway. To model this target discrimination, we use high dynamic range (HDR) natural images to represent 'real-world‘ luminance values that serve as inputs to a biomimetic representation of photoreceptor processing. Adaptive spatiotemporal high-pass filtering (1st-order interneurons) shapes the transient 'edge-like‘ responses, useful for feature discrimination. Following this, a model for the RTC implements a nonlinear facilitation between the rapidly adapting, and independent polarity contrast channels, each with centre-surround antagonism. The recombination of the channels results in increased discrimination of small targets, of approximately the size of a single pixel, without the need for relative motion cues. This method of feature discrimination contrasts with traditional target and background motion-field computations. We show that our RTC-based target detection model is well matched to properties described for the higher-order STMD neurons, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter‘ to successfully detect many targets from the background. The model produces robust target discrimination across a biologically plausible range of target sizes and a range of velocities. We show that the model for small target motion detection is highly correlated to the velocity of the stimulus but not other background statistics, such as local brightness or local contrast, which normally influence target detection tasks. From an engineering perspective, we examine model elaborations for improved target discrimination via inhibitory interactions from correlation-type motion detectors, using a form of antagonism between our feature correlator and the more typical motion correlator. We also observe that a changing optimal threshold is highly correlated to the value of observer ego-motion. We present an elaborated target detection model that allows for implementation of a static optimal threshold, by scaling the target discrimination mechanism with a model-derived velocity estimation of ego-motion. Finally, we investigate the physiological relevance of this target discrimination model. We show that via very subtle image manipulation of the visual stimulus, our model accurately predicts dramatic changes in observed electrophysiological responses from STMD neurons.Thesis (Ph.D.) - University of Adelaide, School of Molecular and Biomedical Science, 200

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modelling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research of insects' visual systems in the literature. These motion perception models or neural networks comprise the looming sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation sensitive neural systems of direction selective neurons (DSNs) in fruit flies, bees and locusts, as well as the small target motion detectors (STMDs) in dragonflies and hover flies. We also review the applications of these models to robots and vehicles. Through these modelling studies, we summarise the methodologies that generate different direction and size selectivity in motion perception. At last, we discuss about multiple systems integration and hardware realisation of these bio-inspired motion perception models

Spatial vision in insects is facilitated by shaping the dynamics of visual input through behavioral action

Egelhaaf M, Boeddeker N, Kern R, Kurtz R, Lindemann JP. Spatial vision in insects is facilitated by shaping the dynamics of visual input through behavioral action. Frontiers in Neural Circuits. 2012;6:108.Insects such as flies or bees, with their miniature brains, are able to control highly aerobatic flight maneuvres and to solve spatial vision tasks, such as avoiding collisions with obstacles, landing on objects, or even localizing a previously learnt inconspicuous goal on the basis of environmental cues. With regard to solving such spatial tasks, these insects still outperform man-made autonomous flying systems. To accomplish their extraordinary performance, flies and bees have been shown by their characteristic behavioral actions to actively shape the dynamics of the image flow on their eyes ("optic flow"). The neural processing of information about the spatial layout of the environment is greatly facilitated by segregating the rotational from the translational optic flow component through a saccadic flight and gaze strategy. This active vision strategy thus enables the nervous system to solve apparently complex spatial vision tasks in a particularly efficient and parsimonious way. The key idea of this review is that biological agents, such as flies or bees, acquire at least part of their strength as autonomous systems through active interactions with their environment and not by simply processing passively gained information about the world. These agent-environment interactions lead to adaptive behavior in surroundings of a wide range of complexity. Animals with even tiny brains, such as insects, are capable of performing extraordinarily well in their behavioral contexts by making optimal use of the closed action-perception loop. Model simulations and robotic implementations show that the smart biological mechanisms of motion computation and visually-guided flight control might be helpful to find technical solutions, for example, when designing micro air vehicles carrying a miniaturized, low-weight on-board processor

Actuation Of Droplets Using Transparent Graphene Electrodes For Tunable Lenses And Biomedical Applications

Variable focal length liquid microlenses are the next candidate for a wide variety of applications. Driving mechanism of the liquid lenses can be categorized into mechanical and electrical actuation. Among different actuation mechanisms, EWOD is the most common tool for actuation of the liquid lenses. In this dissertation, we have demonstrated versatile and low-cost miniature liquid lenses with graphene as electrodes. Tunable focal length is achieved by changing both curvature of the droplet using electrowetting on dielectric (EWOD) and applied pressure. Ionic liquid and KCl solution are utilized as lens liquid on the top of a flexible Teflon-coated PDMS/parylene membrane. Transparent and flexible, graphene allows transmission of visible light as well as large deformation of the polymer membrane to achieve requirements for different lens designs and to increase the field of view without damaging of electrodes. Another advantage of graphene compared to non-transparent electrodes is the larger lens aperture. The tunable range for the focal length is between 3 and 7 mm for a droplet with a volume of 3 μL. The visualization of bone marrow dendritic cells is demonstrated by the liquid lens system with a high resolution (more than 456 lp/mm). The Spherical aberration analysis is performed using COMSOL software to investigate the optical properties of the lens under applied voltages and pressure. We propose a prototype of compound eye with specific design of the electrodes using both tunable lenses and tunable supporting membrane. The design has many advantages including large field of view, compact size and fast response time. This work maybe applicable in the development of the next generation of cameras, endoscopes, cell phones on flexible platform. We also proposed here the design and concept of self-powered wireless sensor based on the graphene radio-frequency (RF) components, which are transparent, flexible, and monolithically integrated on biocompatible soft substrate. We show that a quad-ring circuit based on graphene transistors may simultaneously offer sensing and frequency modulation functions. This battery-free and transparent sensors based on newly discovered 2D nanomaterials may benefit versatile wireless sensing and internet-of-things applications, such as smart contact lenses/glasses and microscope slides

Neural mechanisms underlying target detection in a dragonfly centrifugal neuron

© The Company of Biologists Ltd 2007Visual identification of targets is an important task for many animals searching for prey or conspecifics. Dragonflies utilize specialized optics in the dorsal acute zone, accompanied by higher-order visual neurons in the lobula complex, and descending neural pathways tuned to the motion of small targets. While recent studies describe the physiology of insect small target motion detector (STMD) neurons, little is known about the mechanisms that underlie their exquisite sensitivity to target motion. Lobula plate tangential cells (LPTCs), a group of neurons in dipteran flies selective for wide-field motion, have been shown to take input from local motion detectors consistent with the classic correlation model developed by Hassenstein and Reichardt in the 1950s. We have tested the hypothesis that similar mechanisms underlie the response of dragonfly STMDs. We show that an anatomically characterized centrifugal STMD neuron (CSTMD1) gives responses that depend strongly on target contrast, a clear prediction of the correlation model. Target stimuli are more complex in spatiotemporal terms than the sinusoidal grating patterns used to study LPTCs, so we used a correlation-based computer model to predict response tuning to velocity and width of moving targets. We show that increasing target width in the direction of travel causes a shift in response tuning to higher velocities, consistent with our model. Finally, we show how the morphology of CSTMD1 allows for impressive spatial interactions when more than one target is present in the visual field.Bart R. H. Geurten, Karin Nordström, Jordanna D. H. Sprayberry, Douglas M. Bolzon and David C. O'Carrol

Engineering derivatives from biological systems for advanced aerospace applications

The present study consisted of a literature survey, a survey of researchers, and a workshop on bionics. These tasks produced an extensive annotated bibliography of bionics research (282 citations), a directory of bionics researchers, and a workshop report on specific bionics research topics applicable to space technology. These deliverables are included as Appendix A, Appendix B, and Section 5.0, respectively. To provide organization to this highly interdisciplinary field and to serve as a guide for interested researchers, we have also prepared a taxonomy or classification of the various subelements of natural engineering systems. Finally, we have synthesized the results of the various components of this study into a discussion of the most promising opportunities for accelerated research, seeking solutions which apply engineering principles from natural systems to advanced aerospace problems. A discussion of opportunities within the areas of materials, structures, sensors, information processing, robotics, autonomous systems, life support systems, and aeronautics is given. Following the conclusions are six discipline summaries that highlight the potential benefits of research in these areas for NASA's space technology programs

Biomimetic models of visual navigation - active sensing for embodied intelligence

Insects have developed small scale search behaviours to pursue navigation relevant stimuli more effectively. These often resemble a variation of Zig-Zagging, steering periodically to the left and right, therefore increasing the sampling. In this context we investigate the role of a homologous insect brain structure, the Lateral Accessory Lobe (LAL), which has been described as a pre-motor centre but received limited attention so far. Following a synthesis of the literature on the LAL we developed a steering framework, which proposes that with lateralised stimuli as input, the LAL can initiate a Zig-Zagging behaviour if the input is too weak, meaning unreliable, and targeted steering behaviours if the input is strong, thus reliable. Based on this framework we model a Spiking Neural Network (SNN) investigating a sensory modulated Central Pattern Generator (CPG) as a possible neural mechanism enabling adaptive search behaviours. We investigated the parameter space of the model to discover both the range of possible behaviours as well as which parameter combinations lead to the previously described behaviour. We found that no parameter combination accounts for the majority of observed behaviours. Furthermore, changing the computational noise levels does not lead to break-down of this behaviour. We conclude, that this neural architecture is robust to generate an adaptable Zig-Zagging behaviour. Additionally, we developed a more comprehensive network to explore the functions of known neuron-types with regard to motor control. To investigate how this steering framework might work for view based navigation, we investigated how lateralised sensory input can be used for snapshot navigation. We used a 3D-reconstruction from a LiDAR-scanned field-site (“Antworld”) to generate realistic visual stimuli. Instead of using the entire panorama, we subdivided this into two Fields of View for snapshot generation and the later image comparisons. The difference of image familiarity from both sides was subtracted to initiate a steering response into the most familiar direction. We found that a bigger Field of View alongside non-forward facing memories generated the most correct steering responses towards the snapshot direction. This demonstrates that the LAL-inspired steering framework can be functional for a complex sensori-motor task that had previously not been implicated in LAL functionality. Finally, we modelled how bilateral sensory information and a SNN model of the LAL behave in a snapshot navigation setup using Antworld. We compared the original snapshot navigation model using a panoramic Field of View with several combinations of the Core-Network and bilateral vision models: using a bilateral view, a bilateral view with the SNN, a panoramic view with SNN and other standard movement behaviours. We confirmed the findings of preliminary work, in an abstract setup, that had shown that a bilateral view combined with a SNN performs best to recover and approach navigation relevant locations. Also introducing models based on the steering framework into this visually complex environment improved the performance of agents performing snapshot navigation

NimbleAI: towards neuromorphic sensing-processing 3D-integrated chips

The NimbleAI Horizon Europe project leverages key principles of energy-efficient visual sensing and processing in biological eyes and brains, and harnesses the latest advances in 33D stacked silicon integration, to create an integral sensing-processing neuromorphic architecture that efficiently and accurately runs computer vision algorithms in area-constrained endpoint chips. The rationale behind the NimbleAI architecture is: sense data only with high information value and discard data as soon as they are found not to be useful for the application (in a given context). The NimbleAI sensing-processing architecture is to be specialized after-deployment by tunning system-level trade-offs for each particular computer vision algorithm and deployment environment. The objectives of NimbleAI are: (1) 100x performance per mW gains compared to state-of-the-practice solutions (i.e., CPU/GPUs processing frame-based video); (2) 50x processing latency reduction compared to CPU/GPUs; (3) energy consumption in the order of tens of mWs; and (4) silicon area of approx. 50 mm 2 .NimbleAI has received funding from the EU’s Horizon Europe Research and Innovation programme (Grant Agreement 101070679), and by the UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee (Grant Agreement 10039070)Peer ReviewedArticle signat per 49 autors/es: Xabier Iturbe, IKERLAN, Basque Country (Spain); Nassim Abderrahmane, MENTA, France; Jaume Abella, Barcelona Supercomputing Center (BSC), Catalonia, Spain; Sergi Alcaide, Barcelona Supercomputing Center (BSC), Catalonia, Spain; Eric Beyne, IMEC, Belgium; Henri-Pierre Charles, CEA-LIST, University Grenoble Alpes, France; Christelle Charpin-Nicolle, CEALETI, Univ. Grenoble Alpes, France; Lars Chittka, Queen Mary University of London, UK; Angélica Dávila, IKERLAN, Basque Country (Spain); Arne Erdmann, Raytrix, Germany; Carles Estrada, IKERLAN, Basque Country (Spain); Ander Fernández, IKERLAN, Basque Country (Spain); Anna Fontanelli, Monozukuri (MZ Technologies), Italy; José Flich, Universitat Politecnica de Valencia, Spain; Gianluca Furano, ESA ESTEC, Netherlands; Alejandro Hernán Gloriani, Viewpointsystem, Austria; Erik Isusquiza, ULMA Medical Technologies, Basque Country (Spain); Radu Grosu, TU Wien, Austria; Carles Hernández, Universitat Politecnica de Valencia, Spain; Daniele Ielmini, Politecnico Milano, Italy; David Jackson, University of Manchester, UK; Maha Kooli, CEA-LIST, University Grenoble Alpes, France; Nicola Lepri, Politecnico Milano, Italy; Bernabé Linares-Barranco, CSIC, Spain; Jean-Loup Lachese, MENTA, France; Eric Laurent, MENTA, France; Menno Lindwer, GrAI Matter Labs (GML), Netherlands; Frank Linsenmaier, Viewpointsystem, Austria; Mikel Luján, University of Manchester, UK; Karel Masařík, CODASIP, Czech Republic; Nele Mentens, Universiteit Leiden, Netherlands; Orlando Moreira, GrAI Matter Labs (GML), Netherlands; Chinmay Nawghane, IMEC, Belgium; Luca Peres, University of Manchester, UK; Jean-Philippe Noel, CEA-LIST, University Grenoble Alpes, France; Arash Pourtaherian, GrAI Matter Labs (GML), Netherlands; Christoph Posch, PROPHESEE, France; Peter Priller, AVL List, Austria; Zdenek Prikryl, CODASIP, Czech Republic; Felix Resch, TU Wien, Austria; Oliver Rhodes, University of Manchester, UK; Todor Stefanov, Universiteit Leiden, Netherlands; Moritz Storring, IMEC, Belgium; Michele Taliercio, Monozukuri (MZ Technologies), Italy; Rafael Tornero, Universitat Politecnica de Valencia, Spain; Marcel van de Burgwal, IMEC, Belgium; Geert van der Plas, IMEC, Belgium; Elisa Vianello, CEALETI, Univ. Grenoble Alpes, France; Pavel Zaykov, CODASIP, Czech RepublicPostprint (author's final draft