A 16-Channel Fully Configurable Neural SoC With 1.52 μW/Ch Signal Acquisition, 2.79 μW/Ch Real-Time Spike Classifier, and 1.79 TOPS/W Deep Neural Network Accelerator in 22 nm FDSOI

With the advent of high-density micro-electrodes arrays, developing neural probes satisfying the real-time and stringent power-efficiency requirements becomes more challenging. A smart neural probe is an essential device in future neuroscientific research and medical applications. To realize such devices, we present a 22 nm FDSOI SoC with complex on-chip real-time data processing and training for neural signal analysis. It consists of a digitally-assisted 16-channel analog front-end with 1.52 μ W/Ch, dedicated bio-processing accelerators for spike detection and classification with 2.79 μ W/Ch, and a 125 MHz RISC-V CPU, utilizing adaptive body biasing at 0.5 V with a supporting 1.79 TOPS/W MAC array. The proposed SoC shows a proof-of-concept of how to realize a high-level integration of various on-chip accelerators to satisfy the neural probe requirements for modern applications

GPUs outperform current HPC and neuromorphic solutions in terms of speed and energy when simulating a highly-connected cortical model

While neuromorphic systems may be the ultimate platform for deploying spiking neural networks (SNNs), their distributed nature and optimisation for specific types of models makes them unwieldy tools for developing them. Instead, SNN models tend to be developed and simulated on computers or clusters of computers with standard von Neumann CPU architectures. Over the last decade, as well as becoming a common fixture in many workstations, NVIDIA GPU accelerators have entered the High Performance Computing field and are now used in 50% of the Top 10 super computing sites worldwide. In this paper we use our GeNN code generator to re-implement two neo-cortex-inspired, circuit-scale, point neuron network models on GPU hardware. We verify the correctness of our GPU simulations against prior results obtained with NEST running on traditional HPC hardware and compare the performance with respect to speed and energy consumption against published data from CPU-based HPC and neuromorphic hardware. A full-scale model of a cortical column can be simulated at speeds approaching 0.5× real-time using a single NVIDIA Tesla V100 accelerator – faster than is currently possible using a CPU based cluster or the SpiNNaker neuromorphic system. In addition, we find that, across a range of GPU systems, the energy to solution as well as the energy per synaptic event of the microcircuit simulation is as much as 14× lower than either on SpiNNaker or in CPU-based simulations. Besides performance in terms of speed and energy consumption of the simulation, efficient initialisation of models is also a crucial concern, particularly in a research context where repeated runs and parameter-space exploration are required. Therefore, we also introduce in this paper some of the novel parallel initialisation methods implemented in the latest version of GeNN and demonstrate how they can enable further speed and energy advantages

Accurate, Very Low Computational Complexity Spike Sorting Using Unsupervised Matched Subspace Learning

This paper presents an adaptable dictionary-based feature extraction approach for spike sorting offering high accuracy and low computational complexity for implantable applications. It extracts and learns identifiable features from evolving subspaces through matched unsupervised subspace filtering. To provide compatibility with the strict constraints in implantable devices such as the chip area and power budget, the dictionary contains arrays of {-1, 0 and 1} and the algorithm need only process addition and subtraction operations. Three types of such dictionary were considered. To quantify and compare the performance of the resulting three feature extractors with existing systems, a neural signal simulator based on several different libraries was developed. For noise levels $\sigma_N$ between 0.05 and 0.3 and groups of 3 to 6 clusters, all three feature extractors provide robust high performance with average classification errors of less than 8% over five iterations, each consisting of 100 generated data segments. To our knowledge, the proposed adaptive feature extractors are the first able to classify reliably 6 clusters for implantable applications. An ASIC implementation of the best performing dictionary-based feature extractor was synthesized in a 65-nm CMOS process. It occupies an area of 0.09 mm2 and dissipates up to about 10.48 μW from a 1 V supply voltage, when operating with 8-bit resolution at 30 kHz operating frequency

Photonic neuromorphic information processing and reservoir computing

Photonic neuromorphic computing is attracting tremendous research interest now, catalyzed in no small part by the rise of deep learning in many applications. In this paper, we will review some of the exciting work that has been going in this area and then focus on one particular technology, namely, photonic reservoir computing

Applications of clustering analysis to signal processing problems.

Wing-Keung Sim.Thesis (M.Phil.)--Chinese University of Hong Kong, 1999.Includes bibliographical references (leaves 109-114).Abstracts in English and Chinese.Abstract --- p.2摘要 --- p.3Acknowledgements --- p.4Contents --- p.5List of Figures --- p.8List of Tables --- p.9Introductions --- p.10Chapter 1.1 --- Motivation & Aims --- p.10Chapter 1.2 --- Contributions --- p.11Chapter 1.3 --- Structure of Thesis --- p.11Electrophysiological Spike Discrimination --- p.13Chapter 2.1 --- Introduction --- p.13Chapter 2.2 --- Cellular Physiology --- p.13Chapter 2.2.1 --- Action Potential --- p.13Chapter 2.2.2 --- Recording of Spikes Activities --- p.15Chapter 2.2.3 --- Demultiplexing of Multi-Neuron Recordings --- p.17Chapter 2.3 --- Application of Clustering for Mixed Spikes Train Separation --- p.17Chapter 2.3.1 --- Design Principles for Spike Discrimination Procedures --- p.17Chapter 2.3.2 --- Clustering Analysis --- p.18Chapter 2.3.3 --- Comparison of Clustering Techniques --- p.19Chapter 2.4 --- Literature Review --- p.19Chapter 2.4.1 --- Template Spike Matching --- p.19Chapter 2.4.2 --- Reduced Feature Matching --- p.20Chapter 2.4.3 --- Artificial Neural Networks --- p.21Chapter 2.4.4 --- Hardware Implementation --- p.21Chapter 2.5 --- Summary --- p.22Correlation of Perceived Headphone Sound Quality with Physical Parameters --- p.23Chapter 3.1 --- Introduction --- p.23Chapter 3.2 --- Sound Quality Evaluation --- p.23Chapter 3.3 --- Headphone Characterization --- p.26Chapter 3.3.1 --- Frequency Response --- p.26Chapter 3.3.2 --- Harmonic Distortion --- p.26Chapter 3.3.3 --- Voice-Coil Driver Parameters --- p.27Chapter 3.4 --- Statistical Correlation Measurement --- p.29Chapter 3.4.1 --- Correlation Coefficient --- p.29Chapter 3.4.2 --- t Test for Correlation Coefficients --- p.30Chapter 3.5 --- Summary --- p.31Algorithms --- p.32Chapter 4.1 --- Introduction --- p.32Chapter 4.2 --- Principal Component Analysis --- p.32Chapter 4.2.1 --- Dimensionality Reduction --- p.32Chapter 4.2.2 --- PCA Transformation --- p.33Chapter 4.2.3 --- PCA Implementation --- p.36Chapter 4.3 --- Traditional Clustering Methods --- p.37Chapter 4.3.1 --- Online Template Matching (TM) --- p.37Chapter 4.3.2 --- Online Template Matching Implementation --- p.40Chapter 4.3.3 --- K-Means Clustering --- p.41Chapter 4.3.4 --- K-Means Clustering Implementation --- p.44Chapter 4.4 --- Unsupervised Neural Learning --- p.45Chapter 4.4.1 --- Neural Network Basics --- p.45Chapter 4.4.2 --- Artificial Neural Network Model --- p.46Chapter 4.4.3 --- Simple Competitive Learning (SCL) --- p.47Chapter 4.4.4 --- SCL Implementation --- p.49Chapter 4.4.5 --- Adaptive Resonance Theory Network (ART). --- p.50Chapter 4.4.6 --- ART2 Implementation --- p.53Chapter 4.6 --- Summary --- p.55Experimental Design --- p.57Chapter 5.1 --- Introduction --- p.57Chapter 5.2 --- Electrophysiological Spike Discrimination --- p.57Chapter 5.2.1 --- Experimental Design --- p.57Chapter 5.2.2 --- Extracellular Recordings --- p.58Chapter 5.2.3 --- PCA Feature Extraction --- p.59Chapter 5.2.4 --- Clustering Analysis --- p.59Chapter 5.3 --- Correlation of Headphone Sound Quality with physical Parameters --- p.61Chapter 5.3.1 --- Experimental Design --- p.61Chapter 5.3.2 --- Frequency Response Clustering --- p.62Chapter 5.3.3 --- Additional Parameters Measurement --- p.68Chapter 5.3.4 --- Listening Tests --- p.68Chapter 5.3.5 --- Confirmation Test --- p.69Chapter 5.4 --- Summary --- p.70Results --- p.71Chapter 6.1 --- Introduction --- p.71Chapter 6.2 --- Electrophysiological Spike Discrimination: A Comparison of Methods --- p.71Chapter 6.2.1 --- Clustering Labeled Spike Data --- p.72Chapter 6.2.2 --- Clustering of Unlabeled Data --- p.78Chapter 6.2.3 --- Remarks --- p.84Chapter 6.3 --- Headphone Sound Quality Control --- p.89Chapter 6.3.1 --- Headphones Frequency Response Clustering --- p.89Chapter 6.3.2 --- Listening Tests --- p.90Chapter 6.3.3 --- Correlation with Measured Parameters --- p.90Chapter 6.3.4 --- Confirmation Listening Test --- p.92Chapter 6.4 --- Summary --- p.93Conclusions --- p.97Chapter 7.1 --- Future Work --- p.98Chapter 7.1.1 --- Clustering Analysis --- p.98Chapter 7.1.2 --- Potential Applications of Clustering Analysis --- p.99Chapter 7.2 --- Closing Remarks --- p.100Appendix --- p.101Chapter A.1 --- Tables of Experimental Results: (Spike Discrimination) --- p.101Chapter A.2 --- Tables of Experimental Results: (Headphones Measurement) --- p.104Bibliography --- p.109Publications --- p.11

Racing to Learn: Statistical Inference and Learning in a Single Spiking Neuron with Adaptive Kernels

This paper describes the Synapto-dendritic Kernel Adapting Neuron (SKAN), a simple spiking neuron model that performs statistical inference and unsupervised learning of spatiotemporal spike patterns. SKAN is the first proposed neuron model to investigate the effects of dynamic synapto-dendritic kernels and demonstrate their computational power even at the single neuron scale. The rule-set defining the neuron is simple there are no complex mathematical operations such as normalization, exponentiation or even multiplication. The functionalities of SKAN emerge from the real-time interaction of simple additive and binary processes. Like a biological neuron, SKAN is robust to signal and parameter noise, and can utilize both in its operations. At the network scale neurons are locked in a race with each other with the fastest neuron to spike effectively hiding its learnt pattern from its neighbors. The robustness to noise, high speed and simple building blocks not only make SKAN an interesting neuron model in computational neuroscience, but also make it ideal for implementation in digital and analog neuromorphic systems which is demonstrated through an implementation in a Field Programmable Gate Array (FPGA).Comment: In submission to Frontiers in Neuroscienc

Hardware Architectures and Implementations for Associative Memories : the Building Blocks of Hierarchically Distributed Memories

During the past several decades, the semiconductor industry has grown into a global industry with revenues around $300 billion. Intel no longer relies on only transistor scaling for higher CPU performance, but instead, focuses more on multiple cores on a single die. It has been projected that in 2016 most CMOS circuits will be manufactured with 22 nm process. The CMOS circuits will have a large number of defects. Especially when the transistor goes below sub-micron, the original deterministic circuits will start having probabilistic characteristics. Hence, it would be challenging to map traditional computational models onto probabilistic circuits, suggesting a need for fault-tolerant computational algorithms. Biologically inspired algorithms, or associative memories (AMs)—the building blocks of cortical hierarchically distributed memories (HDMs) discussed in this dissertation, exhibit a remarkable match to the nano-scale electronics, besides having great fault-tolerance ability. Research on the potential mapping of the HDM onto CMOL (hybrid CMOS/nanoelectronic circuits) nanogrids provides useful insight into the development of non-von Neumann neuromorphic architectures and semiconductor industry. In this dissertation, we investigated the implementations of AMs on different hardware platforms, including microprocessor based personal computer (PC), PC cluster, field programmable gate arrays (FPGA), CMOS, and CMOL nanogrids. We studied two types of neural associative memory models, with and without temporal information. In this research, we first decomposed the computational models into basic and common operations, such as matrix-vector inner-product and k-winners-take-all (k-WTA). We then analyzed the baseline performance/price ratio of implementing the AMs with a PC. We continued with a similar performance/price analysis of the implementations on more parallel hardware platforms, such as PC cluster and FPGA. However, the majority of the research emphasized on the implementations with all digital and mixed-signal full-custom CMOS and CMOL nanogrids. In this dissertation, we draw the conclusion that the mixed-signal CMOL nanogrids exhibit the best performance/price ratio over other hardware platforms. We also highlighted some of the trade-offs between dedicated and virtualized hardware circuits for the HDM models. A simple time-multiplexing scheme for the digital CMOS implementations can achieve comparable throughput as the mixed-signal CMOL nanogrids