An investigation into adaptive power reduction techniques for neural hardware

In light of the growing applicability of Artificial Neural Network (ANN) in the signal processing field [1] and the present thrust of the semiconductor industry towards lowpower SOCs for mobile devices [2], the power consumption of ANN hardware has become a very important implementation issue. Adaptability is a powerful and useful feature of neural networks. All current approaches for low-power ANN hardware techniques are ‘non-adaptive’ with respect to the power consumption of the network (i.e. power-reduction is not an objective of the adaptation/learning process). In the research work presented in this thesis, investigations on possible adaptive power reduction techniques have been carried out, which attempt to exploit the adaptability of neural networks in order to reduce the power consumption. Three separate approaches for such adaptive power reduction are proposed: adaptation of size, adaptation of network weights and adaptation of calculation precision. Initial case studies exhibit promising results with significantpower reduction

Neural Network Adaptations to Hardware Implementations

In order to take advantage of the massive parallelism offered by artificial neural networks, hardware implementations are essential. However, most standard neural network models are not very suitable for implementation in hardware and adaptations are needed. In this section an overview is given of the various issues that are encountered when mapping an ideal neural network model onto a compact and reliable neural network hardware implementation, like quantization, handling nonuniformities and nonideal responses, and restraining computational complexity. Furthermore, a broad range of hardware-friendly learning rules is presented, which allow for simpler and more reliable hardware implementations. The relevance of these neural network adaptations to hardware is illustrated by their application in existing hardware implementations

FEEDFORWARD ARTIFICIAL NEURAL NETWORK DESIGN UTILISING SUBTHRESHOLD MODE CMOS DEVICES

This thesis reviews various previously reported techniques for simulating artificial neural networks and investigates the design of fully-connected feedforward networks based on MOS transistors operating in the subthreshold mode of conduction as they are suitable for performing compact, low power, implantable pattern recognition systems. The principal objective is to demonstrate that the transfer characteristic of the devices can be fully exploited to design basic processing modules which overcome the linearity range, weight resolution, processing speed, noise and mismatch of components problems associated with weak inversion conduction, and so be used to implement networks which can be trained to perform practical tasks. A new four-quadrant analogue multiplier, one of the most important cells in the design of artificial neural networks, is developed. Analytical as well as simulation results suggest that the new scheme can efficiently be used to emulate both the synaptic and thresholding functions. To complement this thresholding-synapse, a novel current-to-voltage converter is also introduced. The characteristics of the well known sample-and-hold circuit as a weight memory scheme are analytically derived and simulation results suggest that a dummy compensated technique is required to obtain the required minimum of 8 bits weight resolution. Performance of the combined load and thresholding-synapse arrangement as well as an on-chip update/refresh mechanism are analytically evaluated and simulation studies on the Exclusive OR network as a benchmark problem are provided and indicate a useful level of functionality. Experimental results on the Exclusive OR network and a 'QRS' complex detector based on a 10:6:3 multilayer perceptron are also presented and demonstrate the potential of the proposed design techniques in emulating feedforward neural networks

Configurable Low Power Analog Multilayer Perceptron

A configurable, low power analog implementation of a multilayer perceptron (MLP) is presented in this work. It features a highly programmable system that allows the user to create a MLP neural network design of their choosing. In addition to the configurability, this neural network provides the ability of low power operation via analog circuitry in its neurons. The main MLP system is made up of 12 neurons that can be configurable to any number of layers and neurons per layer until all available resources are utilized. The MLP network is fabricated in a standard 0.13 μm CMOS process occupying approximately 1 mm2 of on-chip area. The MLP system is analyzed at several different configurations with all achieving a greater than 1 Tera-operations per second per Watt figure of merit. This work offers a high speed, low power, and scalable alternative to digital configurable neural networks

Analogue neuromorphic systems.

This thesis addresses a new area of science and technology, that of neuromorphic systems, namely the problems and prospects of analogue neuromorphic systems. The subject is subdivided into three chapters. Chapter 1 is an introduction. It formulates the oncoming problem of the creation of highly computationally costly systems of nonlinear information processing (such as artificial neural networks and artificial intelligence systems). It shows that an analogue technology could make a vital contribution to the creation such systems. The basic principles of creation of analogue neuromorphic systems are formulated. The importance will be emphasised of the principle of orthogonality for future highly efficient complex information processing systems. Chapter 2 reviews the basics of neural and neuromorphic systems and informs on the present situation in this field of research, including both experimental and theoretical knowledge gained up-to-date. The chapter provides the necessary background for correct interpretation of the results reported in Chapter 3 and for a realistic decision on the direction for future work. Chapter 3 describes my own experimental and computational results within the framework of the subject, obtained at De Montfort University. These include: the building of (i) Analogue Polynomial Approximator/lnterpolatoriExtrapolator, (ii) Synthesiser of orthogonal functions, (iii) analogue real-time video filter (performing the homomorphic filtration), (iv) Adaptive polynomial compensator of geometrical distortions of CRT- monitors, (v) analogue parallel-learning neural network (backpropagation algorithm). Thus, this thesis makes a dual contribution to the chosen field: it summarises the present knowledge on the possibility of utilising analogue technology in up-to-date and future computational systems, and it reports new results within the framework of the subject. The main conclusion is that due to its promising power characteristics, small sizes and high tolerance to degradation, the analogue neuromorphic systems will playa more and more important role in future computational systems (in particular in systems of artificial intelligence)