Brain-inspired computing needs a master plan

New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realize this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with the funding, focus and support needed. We have done this in the past with digital technologies; we are in the process of doing it with quantum technologies; can we now do it for brain-inspired computing

Low Cost Interconnected Architecture for the Hardware Spiking Neural Networks

A novel low cost interconnected architecture (LCIA) is proposed in this paper, which is an efficient solution for the neuron interconnections for the hardware spiking neural networks (SNNs). It is based on an all-to-all connection that takes each paired input and output nodes of multi-layer SNNs as the source and destination of connections. The aim is to maintain an efficient routing performance under low hardware overhead. A Networks-on-Chip (NoC) router is proposed as the fundamental component of the LCIA, where an effective scheduler is designed to address the traffic challenge due to irregular spikes. The router can find requests rapidly, make the arbitration decision promptly, and provide equal services to different network traffic requests. Experimental results show that the LCIA can manage the intercommunication of the multi-layer neural networks efficiently and have a low hardware overhead which can maintain the scalability of hardware SNNs

Research on Brain and Mind Inspired Intelligence

To address the problems of scientific theory, common technology and engineering application of multimedia and multimodal information computing, this paper is focused on the theoretical model, algorithm framework, and system architecture of brain and mind inspired intelligence (BMI) based on the structure mechanism simulation of the nervous system, the function architecture emulation of the cognitive system and the complex behavior imitation of the natural system. Based on information theory, system theory, cybernetics and bionics, we define related concept and hypothesis of brain and mind inspired computing (BMC) and design a model and framework for frontier BMI theory. Research shows that BMC can effectively improve the performance of semantic processing of multimedia and cross-modal information, such as target detection, classification and recognition. Based on the brain mechanism and mind architecture, a semantic-oriented multimedia neural, cognitive computing model is designed for multimedia semantic computing. Then a hierarchical cross-modal cognitive neural computing framework is proposed for cross-modal information processing. Furthermore, a cross-modal neural, cognitive computing architecture is presented for remote sensing intelligent information extraction platform and unmanned autonomous system

Аналіз відхилень параметрів електрокардіограм від норми методами машинного навчання

Обʼєктом розгляду роботи є електрична активність серця. Предмет роботи - аналіз ЕКГ методами глибокого навчання. Метою магістерської дисертації є визначення наявності й типу серцевої аритмії у сигналі ЕКГ. В першому розділі розглянуті теоретичні відомості про історію ЕКГ, методику отримання сигналу ЕКГ та можливе їх використання. В другому розділі описується теорія машинного навчання, запропонований метод вирішення поставленого перед роботою завдання. Відповідно запропонованого способу розглянута структура та принципи роботи алгоритмів глибокого навчання згорткового типу. В третьому розділі запропонована структура системи оцінки стану серцевого мʼяза за сигналами ЕКГ. Описані дані, що використовуватимуться та методи їх обробки. Запропонована та реалізована модель глибокого навчання згорткового типу з чотирма прихованими шарами, що згідно з результатами виявилась оптимальною і точною, а саме отримано значення точності 99,96%. Зроблені висновки та запропоновані кроки для майбутнього розвитку наведено у розділі 3.1.The work's object is the heart electric activity. The subject of the work is an ECG analysis by deep learning methods. The purpose of the work is determination of a presence and type of cardiac arrhythmia by the ECG signal. There is overviewed theoretical information about an ECG history, actual extracting and processing methods in the first chapter. The second section describes the theory of machine learning, the proposed method of solving the task. According to the proposed method, the structure and principles of operation of convolutional deep learning algorithms are considered. In the third section, the structure of the system for assessing the condition of the heart muscle based on ECG signals is proposed. The data to be used and the methods of their processing are described. A convolutional type deep learning model with four hidden layers was proposed and implemented, which according to the results turned out to be optimal and accurate, namely, an accuracy value of 99.96% was obtained. There are conclusions and proposed steps for future development announced in section 3.1

NengoFPGA: an FPGA Backend for the Nengo Neural Simulator

Low-power, high-speed neural networks are critical for providing deployable embedded AI applications at the edge. We describe a Xilinx FPGA implementation of Neural Engineering Framework (NEF) networks with online learning that outperforms mobile Nvidia GPU implementations by an order of magnitude or more. Specifically, we provide an embedded Python-capable PYNQ FPGA implementation supported with a Xilinx Vivado High-Level Synthesis (HLS) workflow that allows sub-millisecond implementation of adaptive neural networks with low-latency, direct I/O access to the physical world. The outcome of this work is NengoFPGA, a seamless and user-friendly extension to the neural compiler Python package Nengo. To reduce memory requirements and improve performance we tune the precision of the different intermediate variables in the code to achieve competitive absolute accuracy against slower and larger floating-point reference designs. The online learning component of the neural network exploits immediate feedback to adjust the network weights to best support a given arithmetic precision. As the space of possible design configurations of such quantized networks is vast and is subject to a target accuracy constraint, we use the Hyperopt hyper-parameter tuning tool instead of manual search to find Pareto optimal designs. Specifically, we are able to generate the optimized designs in under 500 short iterations of Vivado HLS C synthesis before running the complete Vivado place-and-route phase on that subset, a much longer process not conducive to rapid exploration. For neural network populations of 64–4096 neurons and 1–8 representational dimensions our optimized FPGA implementation generated by Hyperopt has a speedup of 10–484× over a competing cuBLAS implementation on the Jetson TX1 GPU while using 2.4–9.5× less power. Our speedups are a result of HLS-specific reformulation (15× improvement), precision adaptation (3× improvement), and low-latency direct I/O access (1000× improvement)

Neuromorphic hardware architecture using the neural engineering framework for pattern recognition

We present a hardware architecture that uses the neural engineering framework (NEF) to implement large-scale neural networks on field programmable gate arrays (FPGAS) for performing massively parallel real-time pattern recognition. NEF is a framework that is capable of synthesising large-scale cognitive systems from subnetworks and we have previously presented an FPGA implementation of the NEF that successfully performs nonlinear mathematical computations. That work was developed based on a compact digital neural core, which consists of 64 neurons that are instantiated by a single physical neuron using a time-multiplexing approach. We have now scaled this approach up to build a pattern recognition system by combining identical neural cores together. As a proof of concept, we have developed a handwritten digit recognition system using the MNIST database and achieved a recognition rate of 96.55%. The system is implemented on a state-of-the-art FPGA and can process 5.12 million digits per second. The architecture and hardware optimisations presented offer high-speed and resource-efficient means for performing high-speed, neuromorphic, and massively parallel pattern recognition and classification tasks

Dynamical Systems in Spiking Neuromorphic Hardware

Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case