Design and Implementation of Hardware Accelerators for Neural Processing Applications

Primary motivation for this work was the need to implement hardware accelerators for a newly proposed ANN structure called Auto Resonance Network (ARN) for robotic motion planning. ARN is an approximating feed-forward hierarchical and explainable network. It can be used in various AI applications but the application base was small. Therefore, the objective of the research was twofold: to develop a new application using ARN and to implement a hardware accelerator for ARN. As per the suggestions given by the Doctoral Committee, an image recognition system using ARN has been implemented. An accuracy of around 94% was achieved with only 2 layers of ARN. The network also required a small training data set of about 500 images. Publicly available MNIST dataset was used for this experiment. All the coding was done in Python. Massive parallelism seen in ANNs presents several challenges to CPU design. For a given functionality, e.g., multiplication, several copies of serial modules can be realized within the same area as a parallel module. Advantage of using serial modules compared to parallel modules under area constraints has been discussed. One of the module often useful in ANNs is a multi-operand addition. One problem in its implementation is that the estimation of carry bits when the number of operands changes. A theorem to calculate exact number of carry bits required for a multi-operand addition has been presented in the thesis which alleviates this problem. The main advantage of the modular approach to multi-operand addition is the possibility of pipelined addition with low reconfiguration overhead. This results in overall increase in throughput for large number of additions, typically seen in several DNN configurations

Design and implementation of a digital neural processor for detection applications

The main focus of this research is to develop a digital neural network (processor) and hardware (VLSI) implementation of the same for detection applications, for example in the distance protection of power transmission lines. Using a hardware neural processor will improve the protection system performance over software implementations in terms of speed of operation, response time for faults etc. The main aspects of this research are software design, performance analysis, hardware design and hardware implementation of the digital neural processor. The software design is carried out by developing an object oriented neural network simulator with backpropagation training using C++ language. A preliminary analysis shows that the inputs to the neural network need to be preprocessed. Two filters have been developed for this purpose, based on the analysis of the training data available. The performance analysis involves studying quantization effects (determination of precision requirements) in the network. -- The hardware design involves design of the neural network and the preprocessors. The neural processor consists of three types of processing elements (neurons): input, hidden and output neurons. The input neurons form the input layer of the processor which receive input from the preprocessors. The input layer can be configured to directly receive external input by changing the mode of operation. The output layer gives the signal to the relay for tripping the line under fault. Each neuron consists of datapath and local control unit. Datapath consists of the components for forward and backward passes of the processor and the register file. The local control unit controls the flow of data within a neuron and co-ordinates with the global control unit which controls the flow of data between layers. The neurons and the layers are pipelined for improving the throughput of the processor. The neural processor and the filters are implemented in VLSI using hardware description language (VHDL) and Synopsys / Cadence CAD tools. All the components are individually verified and tested for their functionality and implemented using 0.5 μ CMOS technology

Knowledge Modelling and Learning through Cognitive Networks

One of the most promising developments in modelling knowledge is cognitive network science, which aims to investigate cognitive phenomena driven by the networked, associative organization of knowledge. For example, investigating the structure of semantic memory via semantic networks has illuminated how memory recall patterns influence phenomena such as creativity, memory search, learning, and more generally, knowledge acquisition, exploration, and exploitation. In parallel, neural network models for artificial intelligence (AI) are also becoming more widespread as inferential models for understanding which features drive language-related phenomena such as meaning reconstruction, stance detection, and emotional profiling. Whereas cognitive networks map explicitly which entities engage in associative relationships, neural networks perform an implicit mapping of correlations in cognitive data as weights, obtained after training over labelled data and whose interpretation is not immediately evident to the experimenter. This book aims to bring together quantitative, innovative research that focuses on modelling knowledge through cognitive and neural networks to gain insight into mechanisms driving cognitive processes related to knowledge structuring, exploration, and learning. The book comprises a variety of publication types, including reviews and theoretical papers, empirical research, computational modelling, and big data analysis. All papers here share a commonality: they demonstrate how the application of network science and AI can extend and broaden cognitive science in ways that traditional approaches cannot

A Fast Kohonen Net Implementation for Spert-II

. We present an implementation of Kohonen Self-Organizing Feature Maps for the Spert-II vector microprocessor system. The implementation supports arbitrary neural map topologies and arbitrary neighborhood functions. For small networks, as used in real-world tasks, a single Spert-II board is measured to run Kohonen net classification at up to 208 million connections per second (MCPS). On a speech coding benchmark task, Spert-II performs on-line Kohonen net training at over 100 million connection updates per second (MCUPS). This represents almost a factor of 10 improvement compared to previously reported implementations. The asymptotic peak speed of the system is 213 MCPS and 213 MCUPS. 1 Introduction Spert-II is a workstation accelerator constructed around the T0 vector microprocessor [1]. Although most production use of Spert-II systems has been to accelerate error backpropagation training of multi-layer perceptrons used within continuous speech recognition systems, we designed Spert-..