Mixed-Signal VLSI Implementation of CVNS Artificial Neural Networks

In this work, mixed-signal implementation of Continuous Valued Number System (CVNS) neural network is proposed. The proposed network resolves the limited signal processing precision issue present in mixed-signal neural networks. This is realized by the CVNS addition, the CVNS multiplication and the CVNS sigmoid function evaluation algorithms proposed in this dissertation. The proposed algorithms provide accurate results in low-resolution environment. In addition, an area-efficient low sensitivity CVNS Madaline is proposed. The proposed Madaline is more robust to input and weight errors when compared to the previously developed structures. Moreover, its area consumption is lower. Furthermore, a new approximation scheme for hyperbolic tangent activation function is proposed. Using the proposed approximation scheme results in efficient implementation of digital ASIC neural networks in terms of area, delay and power consumption

A Prototype CVNS Distributed Neural Network

Artificial neural networks are widely used in many applications such as signal processing, classification, and control. However, the practical implementation of them is challenged by the number of inputs, storing the weights, and realizing the activation function.In this work, Continuous Valued Number System (CVNS) distributed neural networks are proposed which are providing the network with self-scaling property. This property aids the network to cope spontaneously with different number of inputs. The proposed CVNS DNN can change the dynamic range of the activation function spontaneously according to the number of inputs providing a proper functionality for the network.In addition, multi-valued CVNS DRAMs are proposed to store the weights as CVNS digits. These memories scan store up to 16 levels, equal to 4 bits, on each storage cell. In addition, they use error correction codes to detect and correct the error over the stored values.A synapse-neuron module is proposed to decrease the design cost. It contains both synapse and neuron and the relevant components. In these modules, the activation function is realized through analog circuits which are far more compact compared to the digital look-up-tables while quite accurate.Furthermore, the redundancy between CVNS digits together with the distributed structure of the neuron make the proposal stable against process violations and reduce the noise to signal ration

Research on the utilization of pattern recognition techniques to identify and classify objects in video data Final report

Spaceborne pattern recognition system for identifying and classifying objects in video dat

Inter-chip communications in an analogue neural network utilising frequency division multiplexing

As advances have been made in semiconductor processing technology, the number of transistors on a chip has increased out of step with the number of input/output pins, which has introduced a communications ’bottle-neck’ in the design of computer architectures. This is a major issue in the hardware design of parallel structures implemented in either digital or analogue VLSI, and is particularly relevant to the design of neural networks which need to be highly interconnected. This work reviews hardware implementations of neural networks, with an emphasis on analogue implementations, and proposes a new method for overcoming connectivity constraints, by the use of Frequency Division Multiplexing (FDM) for the inter-chip communications. In this FDM scheme, multiple analogue signals are transmitted between chips on a single wire by modulating them at different frequencies. The main theoretical work examines the number of signals which can be packed into an FDM channel, depending on the quality factors of the filters used for the demultiplexing, and a fractional overlap parameter which was defined to take into account the inevitable overlapping of filter frequency responses. It is seen that by increasing the amount of permissible overlap, it is possible to communicate a larger number of signals in a given bandwidth. Alternatively, the quality factors of the filters can be reduced, which is advantageous for hardware implementation. Therefore, it was found necessary to determine the amount of overlap which might be permissible in a neural network implementation utilising FDM communications. A software simulator is described, which was designed to test the effects of overlap on Multilayer Perceptron neural networks. Results are presented for networks trained with the backpropagation algorithm, and with the alternative weight perturbation algorithm. These were carried out using both floating point and quantised weights to examine the combined effects of overlap and weight quantisation. It is shown using examples of classification problems, that the neural network learning is indeed highly tolerent to overlap, such that the effect on performance (i.e. on convergence or generalisation) is negligible for fractional overlaps of up to 30%, and some tolerence is achieved for higher overlaps, before failure eventually occurs. The results of the simulations are followed up by a closer examination of the mechanism of network failure. The last section of the thesis investigates the VLSI implementation of the FDM scheme, and proposes the use of the operational transconductance amplifier (OTA) as a building block for implementation of the FDM circuitry in analogue VLSI. A full custom VLSI design of an OTA is presented, which was designed and fabricated through Eurochip, using HSPICE/Mentor Graphics CAD tools and the Mietec 2.4µ CMOS process. A VLSI architecture for inter-chip FDM is also proposed, using adaptive tuning of the OTA-C filters and oscillators.This forms the basis for a program of further work towards the VLSI realisation of inter-chip FDM, which is outlined in the conclusions chapter

Numerical Evaluation of Classification Techniques for Flaw Detection

Nondestructive testing is used extensively throughout the industry for quality assessment and detection of defects in engineering materials. The range and variety of anomalies is enormous and critical assessment of their location and size is often complicated. Depending upon final operational considerations, some of these anomalies may be critical and their detection and classification is therefore of importance. Despite the several advantages of using Nondestructive testing for flaw detection, the conventional NDT techniques based on the heuristic experience-based pattern identification methods have many drawbacks in terms of cost, length and result in erratic analysis and thus lead to discrepancies in results. The use of several statistical and soft computing techniques in the evaluation and classification operations result in the development of an automatic decision support system for defect characterization that offers the possibility of an impartial standardized performance. The present work evaluates the application of both supervised and unsupervised classification techniques for flaw detection and classification in a semi-infinite half space. Finite element models to simulate the MASW test in the presence and absence of voids were developed using the commercial package LS-DYNA. To simulate anomalies, voids of different sizes were inserted on elastic medium. Features for the discrimination of received responses were extracted in time and frequency domains by applying suitable transformations. The compact feature vector is then classified by different techniques: supervised classification (backpropagation neural network, adaptive neuro-fuzzy inference system, k-nearest neighbor classifier, linear discriminate classifier) and unsupervised classification (fuzzy c-means clustering). The classification results show that the performance of k-nearest Neighbor Classifier proved superior when compared with the other techniques with an overall accuracy of 94% in detection of presence of voids and an accuracy of 81% in determining the size of the void in the medium. The assessment of the various classifiers’ performance proved to be valuable in comparing the different techniques and establishing the applicability of simplified classification methods such as k-NN in defect characterization. The obtained classification accuracies for the detection and classification of voids are very encouraging, showing the suitability of the proposed approach to the development of a decision support system for non-destructive testing of materials for defect characterization

Inter-chip communications in an analogue neural network utilising frequency division multiplexing

As advances have been made in semiconductor processing technology, the number of transistors on a chip has increased out of step with the number of input/output pins, which has introduced a communications ’bottle-neck’ in the design of computer architectures. This is a major issue in the hardware design of parallel structures implemented in either digital or analogue VLSI, and is particularly relevant to the design of neural networks which need to be highly interconnected. This work reviews hardware implementations of neural networks, with an emphasis on analogue implementations, and proposes a new method for overcoming connectivity constraints, by the use of Frequency Division Multiplexing (FDM) for the inter-chip communications. In this FDM scheme, multiple analogue signals are transmitted between chips on a single wire by modulating them at different frequencies. The main theoretical work examines the number of signals which can be packed into an FDM channel, depending on the quality factors of the filters used for the demultiplexing, and a fractional overlap parameter which was defined to take into account the inevitable overlapping of filter frequency responses. It is seen that by increasing the amount of permissible overlap, it is possible to communicate a larger number of signals in a given bandwidth. Alternatively, the quality factors of the filters can be reduced, which is advantageous for hardware implementation. Therefore, it was found necessary to determine the amount of overlap which might be permissible in a neural network implementation utilising FDM communications. A software simulator is described, which was designed to test the effects of overlap on Multilayer Perceptron neural networks. Results are presented for networks trained with the backpropagation algorithm, and with the alternative weight perturbation algorithm. These were carried out using both floating point and quantised weights to examine the combined effects of overlap and weight quantisation. It is shown using examples of classification problems, that the neural network learning is indeed highly tolerent to overlap, such that the effect on performance (i.e. on convergence or generalisation) is negligible for fractional overlaps of up to 30%, and some tolerence is achieved for higher overlaps, before failure eventually occurs. The results of the simulations are followed up by a closer examination of the mechanism of network failure. The last section of the thesis investigates the VLSI implementation of the FDM scheme, and proposes the use of the operational transconductance amplifier (OTA) as a building block for implementation of the FDM circuitry in analogue VLSI. A full custom VLSI design of an OTA is presented, which was designed and fabricated through Eurochip, using HSPICE/Mentor Graphics CAD tools and the Mietec 2.4µ CMOS process. A VLSI architecture for inter-chip FDM is also proposed, using adaptive tuning of the OTA-C filters and oscillators.This forms the basis for a program of further work towards the VLSI realisation of inter-chip FDM, which is outlined in the conclusions chapter

A generalised feedforward neural network architecture and its applications to classification and regression

Shunting inhibition is a powerful computational mechanism that plays an important role in sensory neural information processing systems. It has been extensively used to model some important visual and cognitive functions. It equips neurons with a gain control mechanism that allows them to operate as adaptive non-linear filters. Shunting Inhibitory Artificial Neural Networks (SIANNs) are biologically inspired networks where the basic synaptic computations are based on shunting inhibition. SIANNs were designed to solve difficult machine learning problems by exploiting the inherent non-linearity mediated by shunting inhibition. The aim was to develop powerful, trainable networks, with non-linear decision surfaces, for classification and non-linear regression tasks. This work enhances and extends the original SIANN architecture to a more general form called the Generalised Feedforward Neural Network (GFNN) architecture, which contains as subsets both SIANN and the conventional Multilayer Perceptron (MLP) architectures. The original SIANN structure has the number of shunting neurons in the hidden layers equal to the number of inputs, due to the neuron model that is used having a single direct excitatory input. This was found to be too restrictive, often resulting in inadequately small or inordinately large network structures