Inter-chip communications in an analogue neural network utilising frequency division multiplexing

As advances have been made in semiconductor processing technology, the number of transistors on a chip has increased out of step with the number of input/output pins, which has introduced a communications ’bottle-neck’ in the design of computer architectures. This is a major issue in the hardware design of parallel structures implemented in either digital or analogue VLSI, and is particularly relevant to the design of neural networks which need to be highly interconnected. This work reviews hardware implementations of neural networks, with an emphasis on analogue implementations, and proposes a new method for overcoming connectivity constraints, by the use of Frequency Division Multiplexing (FDM) for the inter-chip communications. In this FDM scheme, multiple analogue signals are transmitted between chips on a single wire by modulating them at different frequencies. The main theoretical work examines the number of signals which can be packed into an FDM channel, depending on the quality factors of the filters used for the demultiplexing, and a fractional overlap parameter which was defined to take into account the inevitable overlapping of filter frequency responses. It is seen that by increasing the amount of permissible overlap, it is possible to communicate a larger number of signals in a given bandwidth. Alternatively, the quality factors of the filters can be reduced, which is advantageous for hardware implementation. Therefore, it was found necessary to determine the amount of overlap which might be permissible in a neural network implementation utilising FDM communications. A software simulator is described, which was designed to test the effects of overlap on Multilayer Perceptron neural networks. Results are presented for networks trained with the backpropagation algorithm, and with the alternative weight perturbation algorithm. These were carried out using both floating point and quantised weights to examine the combined effects of overlap and weight quantisation. It is shown using examples of classification problems, that the neural network learning is indeed highly tolerent to overlap, such that the effect on performance (i.e. on convergence or generalisation) is negligible for fractional overlaps of up to 30%, and some tolerence is achieved for higher overlaps, before failure eventually occurs. The results of the simulations are followed up by a closer examination of the mechanism of network failure. The last section of the thesis investigates the VLSI implementation of the FDM scheme, and proposes the use of the operational transconductance amplifier (OTA) as a building block for implementation of the FDM circuitry in analogue VLSI. A full custom VLSI design of an OTA is presented, which was designed and fabricated through Eurochip, using HSPICE/Mentor Graphics CAD tools and the Mietec 2.4µ CMOS process. A VLSI architecture for inter-chip FDM is also proposed, using adaptive tuning of the OTA-C filters and oscillators.This forms the basis for a program of further work towards the VLSI realisation of inter-chip FDM, which is outlined in the conclusions chapter

Inter-chip communications in an analogue neural network utilising frequency division multiplexing

As advances have been made in semiconductor processing technology, the number of transistors on a chip has increased out of step with the number of input/output pins, which has introduced a communications ’bottle-neck’ in the design of computer architectures. This is a major issue in the hardware design of parallel structures implemented in either digital or analogue VLSI, and is particularly relevant to the design of neural networks which need to be highly interconnected. This work reviews hardware implementations of neural networks, with an emphasis on analogue implementations, and proposes a new method for overcoming connectivity constraints, by the use of Frequency Division Multiplexing (FDM) for the inter-chip communications. In this FDM scheme, multiple analogue signals are transmitted between chips on a single wire by modulating them at different frequencies. The main theoretical work examines the number of signals which can be packed into an FDM channel, depending on the quality factors of the filters used for the demultiplexing, and a fractional overlap parameter which was defined to take into account the inevitable overlapping of filter frequency responses. It is seen that by increasing the amount of permissible overlap, it is possible to communicate a larger number of signals in a given bandwidth. Alternatively, the quality factors of the filters can be reduced, which is advantageous for hardware implementation. Therefore, it was found necessary to determine the amount of overlap which might be permissible in a neural network implementation utilising FDM communications. A software simulator is described, which was designed to test the effects of overlap on Multilayer Perceptron neural networks. Results are presented for networks trained with the backpropagation algorithm, and with the alternative weight perturbation algorithm. These were carried out using both floating point and quantised weights to examine the combined effects of overlap and weight quantisation. It is shown using examples of classification problems, that the neural network learning is indeed highly tolerent to overlap, such that the effect on performance (i.e. on convergence or generalisation) is negligible for fractional overlaps of up to 30%, and some tolerence is achieved for higher overlaps, before failure eventually occurs. The results of the simulations are followed up by a closer examination of the mechanism of network failure. The last section of the thesis investigates the VLSI implementation of the FDM scheme, and proposes the use of the operational transconductance amplifier (OTA) as a building block for implementation of the FDM circuitry in analogue VLSI. A full custom VLSI design of an OTA is presented, which was designed and fabricated through Eurochip, using HSPICE/Mentor Graphics CAD tools and the Mietec 2.4µ CMOS process. A VLSI architecture for inter-chip FDM is also proposed, using adaptive tuning of the OTA-C filters and oscillators.This forms the basis for a program of further work towards the VLSI realisation of inter-chip FDM, which is outlined in the conclusions chapter

Can my chip behave like my brain?

Many decades ago, Carver Mead established the foundations of neuromorphic systems. Neuromorphic systems are analog circuits that emulate biology. These circuits utilize subthreshold dynamics of CMOS transistors to mimic the behavior of neurons. The objective is to not only simulate the human brain, but also to build useful applications using these bio-inspired circuits for ultra low power speech processing, image processing, and robotics. This can be achieved using reconfigurable hardware, like field programmable analog arrays (FPAAs), which enable configuring different applications on a cross platform system. As digital systems saturate in terms of power efficiency, this alternate approach has the potential to improve computational efficiency by approximately eight orders of magnitude. These systems, which include analog, digital, and neuromorphic elements combine to result in a very powerful reconfigurable processing machine.Ph.D

Trends and challenges in neuroengineering: toward "Intelligent" neuroprostheses through brain-"brain inspired systems" communication

Future technologies aiming at restoring and enhancing organs function will intimately rely on near-physiological and energy-efficient communication between living and artificial biomimetic systems. Interfacing brain-inspired devices with the real brain is at the forefront of such emerging field, with the term "neurobiohybrids" indicating all those systems where such interaction is established. We argue that achieving a "high-level" communication and functional synergy between natural and artificial neuronal networks in vivo, will allow the development of a heterogeneous world of neurobiohybrids, which will include "living robots" but will also embrace “intelligent” neuroprostheses for augmentation of brain function. The societal and economical impact of intelligent neuroprostheses is likely to be potentially strong, as they will offer novel therapeutic perspectives for a number of diseases, and going beyond classical pharmaceutical schemes. However, they will unavoidably raise fundamental ethical questions on the intermingling between man and machine and more specifically, on how deeply it should be allowed that brain processing is affected by implanted "intelligent" artificial systems. Following this perspective, we provide the reader with insights on ongoing developments and trends in the field of neurobiohybrids. We address the topic also from a "community building" perspective, showing through a quantitative bibliographic analysis, how scientists working on the engineering of brain-inspired devices and brain-machine interfaces are increasing their interactions. We foresee that such trend preludes to a formidable technological and scientific revolution in brain-machine communication and to the opening of new avenues for restoring or even augmenting brain function for therapeutic purposes

Hardware design of LIF with Latency neuron model with memristive STDP synapses

In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode more information than the common Integrate-and-Fire models, typically considered for neuromorphic implementations. In our system LIFL neuron is implemented using CMOS circuits while memristor is used for the implementation of the STDP synapse. A description of the entire circuit is provided. Finally, the capabilities of the proposed architecture have been evaluated by simulating a motif composed of three neurons and two synapses. The simulation results confirm the validity of the proposed system and its suitability for the design of more complex spiking neural network

An Analog Neural Computer with Modular Architecture for Real-Time Dynamic Computations

The paper describes a multichip analog parallel neural network whose architecture, neuron characteristics, synaptic connections, and time constants are modifiable. The system has several important features, such as time constants for time-domain computations, interchangeable chips allowing a modifiable gross architecture, and expandability to any arbitrary size. Such an approach allows the exploration of different network architectures for a wide range of applications, in particular dynamic real-world computations. Four different modules (neuron, synapse, time constant, and switch units) have been designed and fabricated in a 2µm CMOS technology. About 100 of these modules have been assembled in a fully functional prototype neural computer. An integrated software package for setting the network configuration and characteristics, and monitoring the neuron outputs has been developed as well. The performance of the individual modules as well as the overall system response for several applications have been tested successfully. Results of a network for real-time decomposition of acoustical patterns will be discussed