NeuroFlow: A General Purpose Spiking Neural Network Simulation Platform using Customizable Processors

© 2016 Cheung, Schultz and Luk.NeuroFlow is a scalable spiking neural network simulation platform for off-the-shelf high performance computing systems using customizable hardware processors such as Field-Programmable Gate Arrays (FPGAs). Unlike multi-core processors and application-specific integrated circuits, the processor architecture of NeuroFlow can be redesigned and reconfigured to suit a particular simulation to deliver optimized performance, such as the degree of parallelism to employ. The compilation process supports using PyNN, a simulator-independent neural network description language, to configure the processor. NeuroFlow supports a number of commonly used current or conductance based neuronal models such as integrate-and-fire and Izhikevich models, and the spike-timing-dependent plasticity (STDP) rule for learning. A 6-FPGA system can simulate a network of up to ~600,000 neurons and can achieve a real-time performance of 400,000 neurons. Using one FPGA, NeuroFlow delivers a speedup of up to 33.6 times the speed of an 8-core processor, or 2.83 times the speed of GPU-based platforms. With high flexibility and throughput, NeuroFlow provides a viable environment for large-scale neural network simulation

Intrinsically Evolvable Artificial Neural Networks

Dedicated hardware implementations of neural networks promise to provide faster, lower power operation when compared to software implementations executing on processors. Unfortunately, most custom hardware implementations do not support intrinsic training of these networks on-chip. The training is typically done using offline software simulations and the obtained network is synthesized and targeted to the hardware offline. The FPGA design presented here facilitates on-chip intrinsic training of artificial neural networks. Block-based neural networks (BbNN), the type of artificial neural networks implemented here, are grid-based networks neuron blocks. These networks are trained using genetic algorithms to simultaneously optimize the network structure and the internal synaptic parameters. The design supports online structure and parameter updates, and is an intrinsically evolvable BbNN platform supporting functional-level hardware evolution. Functional-level evolvable hardware (EHW) uses evolutionary algorithms to evolve interconnections and internal parameters of functional modules in reconfigurable computing systems such as FPGAs. Functional modules can be any hardware modules such as multipliers, adders, and trigonometric functions. In the implementation presented, the functional module is a neuron block. The designed platform is suitable for applications in dynamic environments, and can be adapted and retrained online. The online training capability has been demonstrated using a case study. A performance characterization model for RC implementations of BbNNs has also been presented

Hardware Implementation of a Visual-Motion Pixel Using Oriented Spatiotemporal Neural Filters

A pixel for measuring two-dimensional (2-D) visual motion with two one-dimensional (1-D) detectors has been implemented in very large scale integration. Based on the spatiotemporal feature extraction model of Adelson and Bergen, the pixel is realized using a general-purpose analog neural computer and a silicon retina. Because the neural computer only offers sum-and-threshold neurons, the Adelson and Bergen\u27s model is modified. The quadratic nonlinearity is replaced with a full-wave rectification, while the contrast normalization is replaced with edge detection and thresholding. Motion is extracted in two dimensions by using two 1-D detectors with spatial smoothing orthogonal to the direction of motion. Analysis shows that our pixel, although it has some limitations, has much lower hardware complexity compared to the full 2-D model. It also produces more accurate results and has a reduced aperture problem compared to the two 1-D model with no smoothing. Real-time velocity is represented as a distribution of activity of the 18 X and 18 Y velocity-tuned neural filter

A Decade of Neural Networks: Practical Applications and Prospects

The Jet Propulsion Laboratory Neural Network Workshop, sponsored by NASA and DOD, brings together sponsoring agencies, active researchers, and the user community to formulate a vision for the next decade of neural network research and application prospects. While the speed and computing power of microprocessors continue to grow at an ever-increasing pace, the demand to intelligently and adaptively deal with the complex, fuzzy, and often ill-defined world around us remains to a large extent unaddressed. Powerful, highly parallel computing paradigms such as neural networks promise to have a major impact in addressing these needs. Papers in the workshop proceedings highlight benefits of neural networks in real-world applications compared to conventional computing techniques. Topics include fault diagnosis, pattern recognition, and multiparameter optimization

Evolution of human computer interaction

The work is devoted to the review of the development the human-computer interaction. In the first sections the history of computing in the "pre-computer" era is briefly described and then the early history of modern computing, methods of the first computers controlling and the tasks of programmers at this stage are described. It describes the methods of interaction with the first -generation computers using the remote control elements, punched cards and punched tapes. The section, devoted to the second generation computers, describes the emergence of high-level operating systems and programming languages. At this point, there are such means of interaction with the computer as the displays and, respectively, such programming tools as interactive languages and interactive debuggers. Research is also beginning on principles of human-computer interaction the infancy of the discipline "computer graphics", the development of computer graphics packages and the emergence of interactive computer graphics standards are considered. In the section “Revolutions in computer science” describes the appearance of a large number of the same series computers and the first super-computers in the context of human-computer interaction. Revolutionary changes are considered in computer graphics and emerging of the science discipline “computer visualization” with its parts “scientific visualization”, “software visualization”, “information visualization” and also “programming by demonstration”. The information about the attempt to create a fifth generation computer based on logical programming is given. It is told about the initial period of teaching programming. The creation of computer networks and the emergence of personal computing as well as the creation the tools of modern parallel computing have become the important stages in the development of modern computing. The virtual reality becomes an important computer visualization tool. The modern state of human-computer interfaces is characterized primarily by emerging of natural interfaces which can be attributed Brain-Computer Interface (Neurocomputer interface, Brain-Computer Interfaces), interfaces based on the direct use of nerve impulses, speech recognition, recognition of lip movement, mimic recognition and eye tracking (Eye Gaze or Eye Tracking), haptic interfaces and also interfaces giving tactile feedback (allowing you to feel the touch),motion capture interfaces the entire human body or individual organs (head, entire arm, hands, fingers, legs), motion capture toolkits,in particular, interfaces based on leg movements (foot-operated computer interfaces), sign interfaces, sign languages. We briefly describe the activity approach to the design of interfaces and also some problems concerning the problem of mass interfaces. Finally, we discuss a number of problems arising from the increasing capabilities of modern computers. The work is in the nature of a popular science article and it largely reflects the subjective impressions of the author. © 2020 National Research Nuclear University. All rights reserved

In-Datacenter Performance Analysis of a Tensor Processing Unit

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.Comment: 17 pages, 11 figures, 8 tables. To appear at the 44th International Symposium on Computer Architecture (ISCA), Toronto, Canada, June 24-28, 201

Методика використання мобільних Інтернет-пристроїв у формуванні загальнопрофесійної складової компетентності бакалавра електромеханіки в моделюванні технічних об'єктів

The article describes the components of methods of using mobile Internet devices in the formation of the general professional component of bachelor in electromechanics competency in modeling of technical objects: using various methods of representing models; solving professional problems using ICT; competence in electric machines and critical thinking. On the content of learning academic disciplines “Higher mathematics”, “Automatic control theory”, “Modeling of electromechanical systems”, “Electrical machines” features of use are disclosed for Scilab, SageCell, Google Sheets, Xcos on Cloud in the formation of the general professional component of bachelor in electromechanics competency in modeling of technical objects. It is concluded that it is advisable to use the following software for mobile Internet devices: a cloud-based spreadsheets as modeling tools (including neural networks), a visual modeling systems as a means of structural modeling of technical objects; a mobile computer mathematical system used at all stages of modeling; a mobile communication tools for organizing joint modeling activities.У статті описані компоненти методики використання мобільних Інтернет-пристроїв при формуванні загально професійної складової компетентності бакалавра електромеханіки в моделюванні технічних об'єктів: використання різних методів подання моделей; розв'язання професійних задач за допомогою ІКТ; компетентність в електричних машинах та критичне мислення. На змісіт навчальних дисциплін "Вища математика", "Теорія автоматичного управління&quot", "Моделювання електромеханічних систем", "Електричні машини"; розкриті особливості використання Scilab, SageCell, Google Sheets, Xcos on Cloud при формуванні загальнопрофесійної складової компетентності бакалавра електромеханіки в моделюванні технічних об'єктів. Зроблено висновок, що доцільно використовувати таке програмне забезпечення мобільних Інтернет-пристроїв: хмаро зорієнтовані електронні таблиці як засіб моделювання (включаючи нейронні мережі), системи візуального моделювання як засіб структурного моделювання технічних об'єктів; мобільні комп'ютерні математичні системи, що використовується на всіх етапах моделювання; мобільні комунікаційні засоби для організації спільної діяльності з моделювання