Brain-inspired Evolutionary Architectures for Spiking Neural Networks

The complex and unique neural network topology of the human brain formed through natural evolution enables it to perform multiple cognitive functions simultaneously. Automated evolutionary mechanisms of biological network structure inspire us to explore efficient architectural optimization for Spiking Neural Networks (SNNs). Instead of manually designed fixed architectures or hierarchical Network Architecture Search (NAS), this paper evolves SNNs architecture by incorporating brain-inspired local modular structure and global cross-module connectivity. Locally, the brain region-inspired module consists of multiple neural motifs with excitatory and inhibitory connections; Globally, we evolve free connections among modules, including long-term cross-module feedforward and feedback connections. We further introduce an efficient multi-objective evolutionary algorithm based on a few-shot performance predictor, endowing SNNs with high performance, efficiency and low energy consumption. Extensive experiments on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS, DVS128-Gesture) demonstrate that our proposed model boosts energy efficiency, archiving consistent and remarkable performance. This work explores brain-inspired neural architectures suitable for SNNs and also provides preliminary insights into the evolutionary mechanisms of biological neural networks in the human brain

舊新迭代之際的文人詠嘆：《名儒草堂詩餘》研究

《名儒草堂詩餘》由元廬陵鳳林書院選錄宋元迭代時期詞作，集萃成書，流芳千古。其收錄詞人多活躍在該時期的江西地帶，故衆多學者認爲讀《名儒草堂詩餘》可以「溯江西詞派」，是研究「宋江西遺民群體」的關竅。 本文將從《名儒草堂詩餘》的收錄詞人與收錄詞作兩方面進行研究。首先從考證詞人群體地緣、交游關係入手，試論該群體「遺民身份」的迥異以及「遁跡經驗」的趨同，並分析收錄詞作的題材偏好以及常見詞調形式與句式，重點闡述三字領在詞作中的作用，探析詞作「發悲愁之」的主旨以及寫作之法，藉以研究詞作風格及《名儒草堂詩餘》的無雙成就

Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks

Spiking Neural Networks (SNNs) are more biologically plausible and computationally efficient. Therefore, SNNs have the natural advantage of drawing the sparse structural plasticity of brain development to alleviate the energy problems of deep neural networks caused by their complex and fixed structures. However, previous SNNs compression works are lack of in-depth inspiration from the brain development plasticity mechanism. This paper proposed a novel method for the adaptive structural development of SNN (SD-SNN), introducing dendritic spine plasticity-based synaptic constraint, neuronal pruning and synaptic regeneration. We found that synaptic constraint and neuronal pruning can detect and remove a large amount of redundancy in SNNs, coupled with synaptic regeneration can effectively prevent and repair over-pruning. Moreover, inspired by the neurotrophic hypothesis, neuronal pruning rate and synaptic regeneration rate were adaptively adjusted during the learning-while-pruning process, which eventually led to the structural stability of SNNs. Experimental results on spatial (MNIST, CIFAR-10) and temporal neuromorphic (N-MNIST, DVS-Gesture) datasets demonstrate that our method can flexibly learn appropriate compression rate for various tasks and effectively achieve superior performance while massively reducing the network energy consumption. Specifically, for the spatial MNIST dataset, our SD-SNN achieves 99.51\% accuracy at the pruning rate 49.83\%, which has a 0.05\% accuracy improvement compared to the baseline without compression. For the neuromorphic DVS-Gesture dataset, 98.20\% accuracy with 1.09\% improvement is achieved by our method when the compression rate reaches 55.50\%

Timestamp Error Detection and Estimation for PMU Data based on Linear Correlation between Relative Phase Angle and Frequency

Time synchronization is essential to synchro-phasor-based applications. However, Timestamp Error (TE) in synchrophasor data can result in application failures. This paper proposes a method for TE detection based on the linear correlation between frequency and relative phase angle. The TE converts the short-term relative phase angle from noise-like signal to one that linear with the frequency. Pearson Correlation Coefficient (PCC) is applied to measure the linear correlation and then detect the timestamp error. The time error is estimated based on the variation of frequency and relative phase angle. Case studies with actual synchrophasor data demonstrate the effectiveness of TE detection and excellent accuracy of TE estimation

Characterization of 35 Masson pine (Pinus massoniana) half-sib families from two provinces based on metabolite properties

Plant metabolism is an important functional trait, and its metabolites have physiological and ecological functions to adapt to the growth environment. However, the physiological and ecological functions of metabolites from different provinces of the same plant species are still unclear. Therefore, this study aimed to determine whether metabolites from different provinces of Masson pine (Pinus massoniana Lamb.) have the corresponding metabolic traits. The gas chromatography–mass spectrometry technique and metabonomic analysis methods were used to characterize 35 Masson pine half-sib families from two provinces. A total of 116 metabolites were putatively identified in 35 families of Masson pine, among which the average content of organic acids was the highest, followed by saccharides and alcohols, and phosphoric acids. Comparative analysis of metabolite groups showed that organic acids, amines, and others were significantly different between the Masson pine families from Guangxi and Guizhou provinces. Six differential metabolites were found between the provinces from Guizhou and Guangxi, namely caffeic acid, L-ascorbic acid, gentiobiose, xylitol, d-pinitol, and β-sitosterol. The most significantly enriched pathways among differentially expressed metabolites between the two provinces were steroid biosynthesis, phenylpropanoid biosynthesis, glutathione metabolism, pentose and glucuronate interconversions. Overall, the results showed that Masson pine half-sib families from different geographical provinces have different metabolite profiles and their metabolites are affected by geographical provenance and growth environment adaptability. This study revealed that the breeding of Masson pine families from different provinces changed the metabolite profiles, providing a reference for the multipurpose breeding of Masson pine

Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks

Spiking Neural Networks (SNNs) have received considerable attention not only for their superiority in energy efficient with discrete signal processing, but also for their natural suitability to integrate multi-scale biological plasticity. However, most SNNs directly adopt the structure of the well-established DNN, rarely automatically design Neural Architecture Search (NAS) for SNNs. The neural motifs topology, modular regional structure and global cross-brain region connection of the human brain are the product of natural evolution and can serve as a perfect reference for designing brain-inspired SNN architecture. In this paper, we propose a Multi-Scale Evolutionary Neural Architecture Search (MSE-NAS) for SNN, simultaneously considering micro-, meso- and macro-scale brain topologies as the evolutionary search space. MSE-NAS evolves individual neuron operation, self-organized integration of multiple circuit motifs, and global connectivity across motifs through a brain-inspired indirect evaluation function, Representational Dissimilarity Matrices (RDMs). This training-free fitness function could greatly reduce computational consumption and NAS's time, and its task-independent property enables the searched SNNs to exhibit excellent transferbility and scalability. Extensive experiments demonstrate that the proposed algorithm achieves state-of-the-art (SOTA) performance with shorter simulation steps on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS and DVS128-Gesture). The thorough analysis also illustrates the significant performance improvement and consistent bio-interpretability deriving from the topological evolution at different scales and the RDMs fitness function

Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

The human brain can self-organize rich and diverse sparse neural pathways to incrementally master hundreds of cognitive tasks. However, most existing continual learning algorithms for deep artificial and spiking neural networks are unable to adequately auto-regulate the limited resources in the network, which leads to performance drop along with energy consumption rise as the increase of tasks. In this paper, we propose a brain-inspired continual learning algorithm with adaptive reorganization of neural pathways, which employs Self-Organizing Regulation networks to reorganize the single and limited Spiking Neural Network (SOR-SNN) into rich sparse neural pathways to efficiently cope with incremental tasks. The proposed model demonstrates consistent superiority in performance, energy consumption, and memory capacity on diverse continual learning tasks ranging from child-like simple to complex tasks, as well as on generalized CIFAR100 and ImageNet datasets. In particular, the SOR-SNN model excels at learning more complex tasks as well as more tasks, and is able to integrate the past learned knowledge with the information from the current task, showing the backward transfer ability to facilitate the old tasks. Meanwhile, the proposed model exhibits self-repairing ability to irreversible damage and for pruned networks, could automatically allocate new pathway from the retained network to recover memory for forgotten knowledge