Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks

Spiking Neural Networks (SNNs) are more biologically plausible and computationally efficient. Therefore, SNNs have the natural advantage of drawing the sparse structural plasticity of brain development to alleviate the energy problems of deep neural networks caused by their complex and fixed structures. However, previous SNNs compression works are lack of in-depth inspiration from the brain development plasticity mechanism. This paper proposed a novel method for the adaptive structural development of SNN (SD-SNN), introducing dendritic spine plasticity-based synaptic constraint, neuronal pruning and synaptic regeneration. We found that synaptic constraint and neuronal pruning can detect and remove a large amount of redundancy in SNNs, coupled with synaptic regeneration can effectively prevent and repair over-pruning. Moreover, inspired by the neurotrophic hypothesis, neuronal pruning rate and synaptic regeneration rate were adaptively adjusted during the learning-while-pruning process, which eventually led to the structural stability of SNNs. Experimental results on spatial (MNIST, CIFAR-10) and temporal neuromorphic (N-MNIST, DVS-Gesture) datasets demonstrate that our method can flexibly learn appropriate compression rate for various tasks and effectively achieve superior performance while massively reducing the network energy consumption. Specifically, for the spatial MNIST dataset, our SD-SNN achieves 99.51\% accuracy at the pruning rate 49.83\%, which has a 0.05\% accuracy improvement compared to the baseline without compression. For the neuromorphic DVS-Gesture dataset, 98.20\% accuracy with 1.09\% improvement is achieved by our method when the compression rate reaches 55.50\%

Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks

Spiking Neural Networks (SNNs) have received considerable attention not only for their superiority in energy efficient with discrete signal processing, but also for their natural suitability to integrate multi-scale biological plasticity. However, most SNNs directly adopt the structure of the well-established DNN, rarely automatically design Neural Architecture Search (NAS) for SNNs. The neural motifs topology, modular regional structure and global cross-brain region connection of the human brain are the product of natural evolution and can serve as a perfect reference for designing brain-inspired SNN architecture. In this paper, we propose a Multi-Scale Evolutionary Neural Architecture Search (MSE-NAS) for SNN, simultaneously considering micro-, meso- and macro-scale brain topologies as the evolutionary search space. MSE-NAS evolves individual neuron operation, self-organized integration of multiple circuit motifs, and global connectivity across motifs through a brain-inspired indirect evaluation function, Representational Dissimilarity Matrices (RDMs). This training-free fitness function could greatly reduce computational consumption and NAS's time, and its task-independent property enables the searched SNNs to exhibit excellent transferbility and scalability. Extensive experiments demonstrate that the proposed algorithm achieves state-of-the-art (SOTA) performance with shorter simulation steps on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS and DVS128-Gesture). The thorough analysis also illustrates the significant performance improvement and consistent bio-interpretability deriving from the topological evolution at different scales and the RDMs fitness function

Factors affecting establishment and population growth of the invasive weed Ambrosia artemisiifolia

Ambrosia artemisiifolia is a highly invasive weed. Identifying the characteristics and the factors influencing its establishment and population growth may help to identify high invasion risk areas and facilitate monitoring and prevention efforts. Six typical habitats: river banks, forests, road margins, farmlands, grasslands, and wastelands, were selected from the main distribution areas of A. artemisiifolia in the Yili Valley, China. Six propagule quantities of A. artemisiifolia at 1, 5, 10, 20, 50, and 100 seeds m-2 were seeded by aggregation, and dispersion in an area without A. artemisiifolia. Using establishment probability models and Allee effect models, we determined the minimum number of seeds and plants required for the establishment and population growth of A. artemisiifolia, respectively. We also assessed the moisture threshold requirements for establishment and survival, and the influence of native species. The influence of propagule pressure on the establishment of A. artemisiifolia was significant. The minimum number of seeds required varied across habitats, with the lowest being 60 seeds m-2 for road margins and the highest being 398 seeds for forests. The minimum number of plants required for population growth in each habitat was 5 and the largest number was 43 in pasture. The aggregation distribution of A. artemisiifolia resulted in a higher establishment and survival rate. The minimum soil volumetric water content required for establishment was significantly higher than that required for survival. The presence of native dominant species significantly reduced the establishment and survival rate of A. artemisiifolia. A. artemisiifolia has significant habitat selectivity and is more likely to establish successfully in a habitat with aggregated seeding with sufficient water and few native species. Establishment requires many seeds but is less affected by the Allee effect after successful establishment, and only a few plants are needed to ensure reproductive success and population growth in the following year. Monitoring should be increased in high invasion risk habitats

Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

The human brain can self-organize rich and diverse sparse neural pathways to incrementally master hundreds of cognitive tasks. However, most existing continual learning algorithms for deep artificial and spiking neural networks are unable to adequately auto-regulate the limited resources in the network, which leads to performance drop along with energy consumption rise as the increase of tasks. In this paper, we propose a brain-inspired continual learning algorithm with adaptive reorganization of neural pathways, which employs Self-Organizing Regulation networks to reorganize the single and limited Spiking Neural Network (SOR-SNN) into rich sparse neural pathways to efficiently cope with incremental tasks. The proposed model demonstrates consistent superiority in performance, energy consumption, and memory capacity on diverse continual learning tasks ranging from child-like simple to complex tasks, as well as on generalized CIFAR100 and ImageNet datasets. In particular, the SOR-SNN model excels at learning more complex tasks as well as more tasks, and is able to integrate the past learned knowledge with the information from the current task, showing the backward transfer ability to facilitate the old tasks. Meanwhile, the proposed model exhibits self-repairing ability to irreversible damage and for pruned networks, could automatically allocate new pathway from the retained network to recover memory for forgotten knowledge

Nutrient addition affects leaf N-P scaling relationship in Arabidopsis thaliana

Ambient nutrient changes influence the coupling of nitrogen (N) and phosphorus (P) in terrestrial ecosystems, but whether it could alter the scaling relationship of plant leaf N to P concentrations remains unclear. Moreover, knowledge about how multi-elemental stoichiometry responds to varying N and P availabilities remains limited. Here we conducted experimental manipulations using Arabidopsis thaliana, with five N and P addition levels and nine repeated experiments. We found that the scaling exponent of leaf N to P concentrations decreased with increasing N levels, but increased with increasing P levels. This suggests that high nutrient availability decreases the variability of its own concentration, but promotes the fluctuation in another tightly associated nutrient concentration in leaves among plant individuals. We call this as Nutrient Availability–Individual Variability Hypothesis. In addition, N and P supply exerted differential influences on the concentrations of multi-elements in leaves. Compared with the green-leaves, the senesced-leaves had higher variability of C, N, P, K and Mg concentrations but lower variability of Ca concentration under varying nutrient availabilities. This suggests that stage-dependent pattern of leaf stoichiometric homeostasis relies on the type of elements, and the elemental feature should be considered when choosing a more favorable tissue in plants for diagnosing the nutrient availability in ambient environments. These findings provide a novel mechanism for understanding the dynamic processes of population structure and functioning under global nutrient changes, which should be incorporated into modeling stoichiometric growth in terrestrial ecosystems. Furthermore, our study can advance the holistic understanding about plant eco-physiological response and adaption under global nutrient changes from the stoichiometric perspective of multiple elements beyond N and P

Kick Bad Guys Out! Zero-Knowledge-Proof-Based Anomaly Detection in Federated Learning

Federated learning (FL) systems are vulnerable to malicious clients that submit poisoned local models to achieve their adversarial goals, such as preventing the convergence of the global model or inducing the global model to misclassify some data. Many existing defense mechanisms are impractical in real-world FL systems, as they require prior knowledge of the number of malicious clients or rely on re-weighting or modifying submissions. This is because adversaries typically do not announce their intentions before attacking, and re-weighting might change aggregation results even in the absence of attacks. To address these challenges in real FL systems, this paper introduces a cutting-edge anomaly detection approach with the following features: i) Detecting the occurrence of attacks and performing defense operations only when attacks happen; ii) Upon the occurrence of an attack, further detecting the malicious client models and eliminating them without harming the benign ones; iii) Ensuring honest execution of defense mechanisms at the server by leveraging a zero-knowledge proof mechanism. We validate the superior performance of the proposed approach with extensive experiments

Automatic Visual Inspection and Condition-Based Maintenance for Catenary

Defects on catenary components are a major part of device faults as a result of a much higher tension on high-speed catenary, such as looseness of bolts, component broken, and component missing. Traditional inspection on catenary components has to be performed only at night with human eyes. Not only the inspection speed is very slow but also the inspection results are not reliable, as a result of the poor lighting environment and long-time working tiredness. In this chapter, we present an automatic visual inspection system for checking the status of components on catenary. A dedicated designed camera system is mounted on an inspection car, which covers almost all the components to be checked and gives great details of each component. Considering the great data storm at each catenary post, high-performance servers with GPU acceleration are used, and technologies of multi-thread and parallel computing are exploited. Furthermore, an intelligent analysis framework is proposed, which uses structural analysis to localize each component in the image and perform automatic detection based on different features such as geometry, texture, and logic rules. The system has been successfully used in China’s high-speed railways, which shows great advantages in the catenary inspection application

Self-Powered Stretchable Sensor Arrays Exhibiting Magnetoelasticity for Real-Time Human–Machine Interaction

Stretchable strain sensors are highly desirable for human motion monitoring, and can be used to build new forms of bionic robots. However, the current use of flexible strain gauges is hindered by the need for an external power supply, and the demand for long-term operation. Here, a new flexible self-powered strain sensor system based on an electromagnetic generator that possesses a high stretchability in excess of 150%, a short response time of 30 ms, and an excellent linearity (R2 &gt; 0.98), is presented. Based on this new form of sensor, a human–machine interaction system is designed to achieve remote control of a robot hand and vehicle using a human hand, which provides a new scheme for real-time gesture interaction.</p

A Semi-supervised Sensing Rate Learning based CMAB Scheme to Combat COVID-19 by Trustful Data Collection in the Crowd

Mobile CrowdSensing (MCS), through employing considerable workers to sense and collect data in a participatory manner, has been recognized as a promising paradigm for building many large-scale applications in a cost-effective way, such as combating COVID-19. The recruitment of trustworthy and high-quality workers is an important research issue for MCS. Previous studies assume that the qualities of workers are known in advance, or the platform knows the qualities of workers once it receives their collected data. In reality, to reduce their costs and thus maximize revenue, many strategic workers do not perform their sensing tasks honestly and report fake data to the platform. So, it is very hard for the platform to evaluate the authenticity of the received data. In this paper, an incentive mechanism named Semi-supervision based Combinatorial Multi-Armed Bandit reverse Auction (SCMABA) is proposed to solve the recruitment problem of multiple unknown and strategic workers in MCS. First, we model the worker recruitment as a multi-armed bandit reverse auction problem, and design an UCB-based algorithm to separate the exploration and exploitation, considering the Sensing Rates (SRs) of recruited workers as the gain of the bandit. Next, a Semi-supervised Sensing Rate Learning (SSRL) approach is proposed to quickly and accurately obtain the workers' SRs, which consists of two phases, supervision and self-supervision. Last, SCMABA is designed organically combining the SRs acquisition mechanism with multi-armed bandit reverse auction, where supervised SR learning is used in the exploration, and the self-supervised one is used in the exploitation. We prove that our SCMABA achieves truthfulness and individual rationality. Additionally, we exhibit outstanding performances of the SCMABA mechanism through in-depth simulations of real-world data traces.Comment: 18 pages, 14 figure