Spiking NeRF: Making Bio-inspired Neural Networks See through the Real World

Spiking neuron networks (SNNs) have been thriving on numerous tasks to leverage their promising energy efficiency and exploit their potentialities as biologically plausible intelligence. Meanwhile, the Neural Radiance Fields (NeRF) render high-quality 3D scenes with massive energy consumption, and few works delve into the energy-saving solution with a bio-inspired approach. In this paper, we propose spiking NeRF (SpikingNeRF), which aligns the radiance ray with the temporal dimension of SNN, to naturally accommodate the SNN to the reconstruction of Radiance Fields. Thus, the computation turns into a spike-based, multiplication-free manner, reducing the energy consumption. In SpikingNeRF, each sampled point on the ray is matched onto a particular time step, and represented in a hybrid manner where the voxel grids are maintained as well. Based on the voxel grids, sampled points are determined whether to be masked for better training and inference. However, this operation also incurs irregular temporal length. We propose the temporal condensing-and-padding (TCP) strategy to tackle the masked samples to maintain regular temporal length, i.e., regular tensors, for hardware-friendly computation. Extensive experiments on a variety of datasets demonstrate that our method reduces the $76.74\%$ energy consumption on average and obtains comparable synthesis quality with the ANN baseline

Dynamical systems for dealing with classification tasks in industrial environments

Intelligent techniques that emulate characteristics of biological systems offer opportunities for industrial applications with new and interesting capabilities. In the competitive economic environment these control techniques can provide products with high competitive values. The technique shown in this paper, based in bio-inspired neural networks, is used to illustrate these principles

Koniocortex-like network unsupervised learning surpasses supervised results on WBCD breast cancer database

Koniocortex-Like Network is a novel category of Bio-Inspired Neural Networks whose architecture and properties are inspired in the biological koniocortex, the ?rst layer of the cortex that receives information from the thalamus. In the Koniocortex-Like Network competition and pattern classi?cation emerges naturally due to the interplay of inhibitory interneurons, metaplasticity and intrinsic plasticity. Recently proposed, it has shown a big potential for complex tasks with unsupervised learning. Now for the ?rst time, its competitive results are proved in a relevant standard real application that is the objective of state-ofthe-art research: the diagnosis of breast cancer data from the Wisconsin Breast Cancer Databas

Bio-inspired Neural Networks for Angular Velocity Estimation in Visually Guided Flights

Executing delicate flight maneuvers using visual information is a huge challenge for future robotic vision systems. As a source of inspiration, insects are quite apt at navigating in woods and landing on surfaces which require delicate visual perception and flight control. The exquisite sensitivity of insects for image motion speed, as revealed recently, is coming from a class of specific neurons called descending neurons. Some of the descending neurons have demonstrated angular velocity selectivity as the image motion speed varies in retina. Build a quantitative angular velocity detection model is the first step for not only further understanding of the biological visual system, but also providing robust and economic solutions of visual motion perception for an artificial visual system. This thesis aims to explore biological image processing methods for motion speed detection in visually guided flights. The major contributions are summarized as follows. We have presented an angular velocity decoding model (AVDM), which estimates the visual motion speed combining both textural and temporal information from input signals. The model consists of three parts: elementary motion detection circuits, wide-field texture estimation pathway and angular velocity decoding layer. The model estimates the angular velocity very well with improved spatial frequency independence compared to the state-of-the-art angular velocity detecting models, when firstly tested by moving sinusoidal gratings. This spatial independence is vital to account for the honeybee’s flight behaviors. We have also investigated the spatial and temporal resolutions of honeybees to get a bio-plausible parameter setting for explaining these behaviors. To investigate whether the model can account for observations of tunnel centering behaviors of honeybees, the model has been implemented in a virtual bee simulated by the game engine Unity. The simulation results of a series of experiments show that the agent can adjust its position to fly through patterned tunnels by balancing the angular velocities estimated on both eyes under several circumstances. All tunnel stimulations reproduce similar behaviors of real bees, which indicate that our model does provide a possible explanation for estimating the image velocity and can be used for MAV’s flight course regulation in tunnels. What’s more, to further verify the robustness of the model, the visually guided terrain following simulations have been carried out with a closed-loop control scheme to restore a preset angular velocity during the flight. The simulation results of successfully flying over the undulating terrain verify the feasibility and robustness of the AVDM performing in various application scenarios, which shows its potential in applications of micro aerial vehicle’s terrain following. In addition, we have also applied the AVDM in grazing landing using only visual information. A LGMD neuron is also introduced to avoid collision and to trigger the hover phase, which ensures the safety of landing. By applying honeybee’s landing strategy of keeping constant angular velocity, we have designed a close-loop control scheme with an adaptive gain to control landing dynamic using AVDM response as input. A series of controlled trails have been designed in Unity platform to demonstrate the effectiveness of the proposed model and control scheme for visual landing under various conditions. The proposed model could be implemented into real small robots to investigate the robustness in real landing scenarios in near future

Glasius bio-inspired neural networks based UV-C disinfection path planning improved by preventive deadlock processing algorithm

The COVID-19 pandemic made robot manufacturers explore the idea of combining mobile robotics with UV-C light to automate the disinfection processes. But performing this process in an optimum way introduces some challenges: on the one hand, it is necessary to guarantee that all surfaces receive the radiation level to ensure the disinfection; at the same time, it is necessary to minimize the radiation dose to avoid the damage of the environment. In this work, both challenges are addressed with the design of a complete coverage path planning (CCPP) algorithm. To do it, a novel architecture that combines the glasius bio-inspired neural network (GBNN), a motion strategy, an UV-C estimator, a speed controller, and a pure pursuit controller have been designed. One of the main issues in CCPP is the deadlocks. In this application they may cause a loss of the operation, lack of regularity and high peaks in the radiation dose map, and in the worst case, they can make the robot to get stuck and not complete the disinfection process. To tackle this problem, in this work we propose a preventive deadlock processing algorithm (PDPA) and an escape route generator algorithm (ERGA). Simulation results show how the application of PDPA and the ERGA allow to complete complex maps in an efficient way where the application of GBNN is not enough. Indeed, a 58% more of covered surface is observed. Furthermore, two different motion strategies have been compared: boustrophedon and spiral motion, to check its influence on the performance of the robot navigation

LGMD based neural network for automatic collision detection

Real-time collision detection in dynamic scenarios is a hard task if the algorithms used are based on conventional techniques of computer vision, since these are computationally complex and, consequently, time-consuming. On the other hand, bio-inspired visual sensors are suitable candidates for mobile robot navigation in unknown environments, due to their computational simplicity. The Lobula Giant Movement Detector (LGMD) neuron, located in the locust optic lobe, responds selectively to approaching objects. This neuron has been used to develop bio-inspired neural networks for collision avoidance. In this work, we propose a new LGMD model based on two previous models, in order to improve over them by incorporating other algorithms. To assess the real-time properties of the proposed model, it was applied to a real robot. Results shown that the LGMD neuron model can robustly support collision avoidance in complex visual scenarios.(undefined

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

Spiking neural networks (SNNs) are bio-inspired neural networks with asynchronous discrete and sparse characteristics, which have increasingly manifested their superiority in low energy consumption. Recent research is devoted to utilizing spatio-temporal information to directly train SNNs by backpropagation. However, the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing and network degradation, which greatly limits the performance of directly trained SNNs and prevents them from going deeper. In this paper, we propose a multi-level firing (MLF) method based on the existing spatio-temporal back propagation (STBP) method, and spiking dormant-suppressed residual network (spiking DS-ResNet). MLF enables more efficient gradient propagation and the incremental expression ability of the neurons. Spiking DS-ResNet can efficiently perform identity mapping of discrete spikes, as well as provide a more suitable connection for gradient propagation in deep SNNs. With the proposed method, our model achieves superior performances on a non-neuromorphic dataset and two neuromorphic datasets with much fewer trainable parameters and demonstrates the great ability to combat the gradient vanishing and degradation problem in deep SNNs.Comment: Accepted by the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22

A multirobot platform based on autonomous surface and underwater vehicles with bio-inspired neurocontrollers for long-term oil spills monitoring

This paper describes the BUSCAMOS-Oil monitoring system, which is a robotic platform consisting of an autonomous surface vessel combined with an underwater vehicle. The system has been designed for the long-term monitoring of oil spills, including the search for the spill, and transmitting information on its location, extent, direction and speed. Both vehicles are controlled by two different types of bio-inspired neural networks: a Self-Organization Direction Mapping Network for trajectory generation and a Neural Network for Avoidance Behaviour for avoiding obstacles. The systems’ resilient capabilities are provided by bio-inspired algorithms implemented in a modular software architecture and controlled by redundant devices to give the necessary robustness to operate in the difficult conditions typically found in long-term oil-spill operations. The efficacy of the vehicles’ adaptive navigation system and long-term mission capabilities are shown in the experimental results.This work was partially supported by the BUSCAMOS Project (ref. 1003211003700) under the program DN8644 COINCIDENTE of the Spanish Defense Ministry, the “Research Programme for Groups of Scientific Excellence at Region of Murcia” of the Seneca Foundation (Agency for Science and Technology of the Region of Murcia-19895/GERM/15)”, and the Spanish Government’s cDrone (ref. TIN2013-45920-R) and ViSelTR (ref. TIN2012-39279) projects

Efficient hardware implementations of bio-inspired networks

The human brain, with its massive computational capability and power efficiency in small form factor, continues to inspire the ultimate goal of building machines that can perform tasks without being explicitly programmed. In an effort to mimic the natural information processing paradigms observed in the brain, several neural network generations have been proposed over the years. Among the neural networks inspired by biology, second-generation Artificial or Deep Neural Networks (ANNs/DNNs) use memoryless neuron models and have shown unprecedented success surpassing humans in a wide variety of tasks. Unlike ANNs, third-generation Spiking Neural Networks (SNNs) closely mimic biological neurons by operating on discrete and sparse events in time called spikes, which are obtained by the time integration of previous inputs. Implementation of data-intensive neural network models on computers based on the von Neumann architecture is mainly limited by the continuous data transfer between the physically separated memory and processing units. Hence, non-von Neumann architectural solutions are essential for processing these memory-intensive bio-inspired neural networks in an energy-efficient manner. Among the non-von Neumann architectures, implementations employing non-volatile memory (NVM) devices are most promising due to their compact size and low operating power. However, it is non-trivial to integrate these nanoscale devices on conventional computational substrates due to their non-idealities, such as limited dynamic range, finite bit resolution, programming variability, etc. This dissertation demonstrates the architectural and algorithmic optimizations of implementing bio-inspired neural networks using emerging nanoscale devices. The first half of the dissertation focuses on the hardware acceleration of DNN implementations. A 4-layer stochastic DNN in a crossbar architecture with memristive devices at the cross point is analyzed for accelerating DNN training. This network is then used as a baseline to explore the impact of experimental memristive device behavior on network performance. Programming variability is found to have a critical role in determining network performance compared to other non-ideal characteristics of the devices. In addition, noise-resilient inference engines are demonstrated using stochastic memristive DNNs with 100 bits for stochastic encoding during inference and 10 bits for the expensive training. The second half of the dissertation focuses on a novel probabilistic framework for SNNs using the Generalized Linear Model (GLM) neurons for capturing neuronal behavior. This work demonstrates that probabilistic SNNs have comparable perform-ance against equivalent ANNs on two popular benchmarks - handwritten-digit classification and human activity recognition. Considering the potential of SNNs in energy-efficient implementations, a hardware accelerator for inference is proposed, termed as Spintronic Accelerator for Probabilistic SNNs (SpinAPS). The learning algorithm is optimized for a hardware friendly implementation and uses first-to-spike decoding scheme for low latency inference. With binary spintronic synapses and digital CMOS logic neurons for computations, SpinAPS achieves a performance improvement of 4x in terms of GSOPS/W/mm$^2$ when compared to a conventional SRAM-based design. Collectively, this work demonstrates the potential of emerging memory technologies in building energy-efficient hardware architectures for deep and spiking neural networks. The design strategies adopted in this work can be extended to other spike and non-spike based systems for building embedded solutions having power/energy constraints