A modified neural network model for Lobula Giant Movement Detector with additional depth movement feature

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron that is located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement

Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background

The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is filtered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: first, the neural network fulfilled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector

A modified model for the Lobula Giant Movement Detector and its FPGA implementation

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector

Reactive direction control for a mobile robot: A locust-like control of escape direction emerges when a bilateral pair of model locust visual neurons are integrated

Locusts possess a bilateral pair of uniquely identifiable visual neurons that respond vigorously to the image of an approaching object. These neurons are called the lobula giant movement detectors (LGMDs). The locust LGMDs have been extensively studied and this has lead to the development of an LGMD model for use as an artificial collision detector in robotic applications. To date, robots have been equipped with only a single, central artificial LGMD sensor, and this triggers a non-directional stop or rotation when a potentially colliding object is detected. Clearly, for a robot to behave autonomously, it must react differently to stimuli approaching from different directions. In this study, we implement a bilateral pair of LGMD models in Khepera robots equipped with normal and panoramic cameras. We integrate the responses of these LGMD models using methodologies inspired by research on escape direction control in cockroaches. Using ‘randomised winner-take-all’ or ‘steering wheel’ algorithms for LGMD model integration, the khepera robots could escape an approaching threat in real time and with a similar distribution of escape directions as real locusts. We also found that by optimising these algorithms, we could use them to integrate the left and right DCMD responses of real jumping locusts offline and reproduce the actual escape directions that the locusts took in a particular trial. Our results significantly advance the development of an artificial collision detection and evasion system based on the locust LGMD by allowing it reactive control over robot behaviour. The success of this approach may also indicate some important areas to be pursued in future biological research

Near range path navigation using LGMD visual neural networks

In this paper, we proposed a method for near range path navigation for a mobile robot by using a pair of biologically inspired visual neural network – lobula giant movement detector (LGMD). In the proposed binocular style visual system, each LGMD processes images covering a part of the wide field of view and extracts relevant visual cues as its output. The outputs from the two LGMDs are compared and translated into executable motor commands to control the wheels of the robot in real time. Stronger signal from the LGMD in one side pushes the robot away from this side step by step; therefore, the robot can navigate in a visual environment naturally with the proposed vision system. Our experiments showed that this bio-inspired system worked well in different scenarios

Redundant neural vision systems: competing for collision recognition roles

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modelling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems – the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modelling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research of insects' visual systems in the literature. These motion perception models or neural networks comprise the looming sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation sensitive neural systems of direction selective neurons (DSNs) in fruit flies, bees and locusts, as well as the small target motion detectors (STMDs) in dragonflies and hover flies. We also review the applications of these models to robots and vehicles. Through these modelling studies, we summarise the methodologies that generate different direction and size selectivity in motion perception. At last, we discuss about multiple systems integration and hardware realisation of these bio-inspired motion perception models

Bio-inspired Collision Detection with Motion Cues Enhancement in Dim Light Environments

Detecting looming objects robustly and timely is a huge challenge for artificial vision systems in complex natural scenes, including dim light scenes. Insects have evolved remarkable capacities in collision detection despite their tiny eyes and brains. The locusts’ LGMD1 neuron shows strong looming-sensitive property for both light and dark objects, which is a source of inspiration for developing collision detection systems. Furthermore, specialized visual processing strategies in nocturnal animals’ brains can provide inspiration for detecting faint motion like dim-light collision detection when challenged with low light conditions. This research aims to explore theLGMD1 based collision detection methods, adaptive low-light image enhancement methods, biologically-inspired solutions for enhancing faint motion cues as well as collision detection methods in low light conditions. The major contributions are summarized as follows. A new visual neural system model (LGMD1) is developed, which applies a neural competition mechanism within a framework of separated ON and OFF pathways to shut off the translating response. The competition-based approach responds vigorously to monotonous ON/OFF responses resulting from a looming object. However, it does not respond to paired ON-OFF responses that result from a translating object, thereby enhancing collision selectivity. Moreover, a complementary denoising mechanism ensures reliable collision detection. To verify the effectiveness of the model, we have conducted systematic comparative experiments on synthetic and real datasets. The results show that our method exhibits more accurate discrimination between looming and translational events—the looming motion can be correctly detected. It also demonstrates that the proposed model is more robust than comparative models. A framework is proposed for adaptively enhancing low-light images, which implements the processing of dark adaptation with proper adaptation parameters in R,G and B channels separately. Specifically, the dark adaptation processing consists of a series of canonical neural computations, including the power law adaptation, divisive normalization and adaptive rescaling operations. Experimental results show that the proposed bioinspired dark adaptation framework is more efficient and can better preserve the naturalness of the image compared with several representative low light image enhancement methods. A dim-light motion cues enhancement (DLMCE) model is designed for extracting extremely faint motion cues. This model integrates dark-adaptation, spatio-temporal constraint and neural summation mechanisms, which are achieved with canonical neural computations and neural summation in temporal and spatial domains, to enhance faint motion cues. With the DLMCE model, the image intensity and contrast are first increased by the dark adaptation processing, then the strong motion cues are extracted by the spatio-temporal constraint strategy, and these motion cues are further enhanced by neural summation mechanisms. Experimental results have demonstrated that the presented DLMCE model outperforms the existing methods for dim-light motion cues enhancement, and faint motion cues can be successfully detected in consecutive frames efficiently. As demonstrated in the experiments, the proposed DLMCE model provides a robust and effective solution for autonomous systems in detecting moving objects under low light conditions. A bio-inspired collision detection model is developed for detecting looming objects in dim light environments. The model combines the DLMCE model with the classical four-layered LGMD1 model to detect dimly illuminated approaching objects. To verify the effectiveness of the model, we have conducted comparative experiments on real looming datasets. The results have demonstrated that the proposed bio-inspired collision detection model can correctly recognize looming objects under low light conditions since the DLMCE model enhances the faint looming cues