Strategies for neural networks in ballistocardiography with a view towards hardware implementation

A thesis submitted for the degree of Doctor of Philosophy at the University of LutonThe work described in this thesis is based on the results of a clinical trial conducted by the research team at the Medical Informatics Unit of the University of Cambridge, which show that the Ballistocardiogram (BCG) has prognostic value in detecting impaired left ventricular function before it becomes clinically overt as myocardial infarction leading to sudden death. The objective of this study is to develop and demonstrate a framework for realising an on-line BCG signal classification model in a portable device that would have the potential to find pathological signs as early as possible for home health care. Two new on-line automatic BeG classification models for time domain BeG classification are proposed. Both systems are based on a two stage process: input feature extraction followed by a neural classifier. One system uses a principal component analysis neural network, and the other a discrete wavelet transform, to reduce the input dimensionality. Results of the classification, dimensionality reduction, and comparison are presented. It is indicated that the combined wavelet transform and MLP system has a more reliable performance than the combined neural networks system, in situations where the data available to determine the network parameters is limited. Moreover, the wavelet transfonn requires no prior knowledge of the statistical distribution of data samples and the computation complexity and training time are reduced. Overall, a methodology for realising an automatic BeG classification system for a portable instrument is presented. A fully paralJel neural network design for a low cost platform using field programmable gate arrays (Xilinx's XC4000 series) is explored. This addresses the potential speed requirements in the biomedical signal processing field. It also demonstrates a flexible hardware design approach so that an instrument's parameters can be updated as data expands with time. To reduce the hardware design complexity and to increase the system performance, a hybrid learning algorithm using random optimisation and the backpropagation rule is developed to achieve an efficient weight update mechanism in low weight precision learning. The simulation results show that the hybrid learning algorithm is effective in solving the network paralysis problem and the convergence is much faster than by the standard backpropagation rule. The hidden and output layer nodes have been mapped on Xilinx FPGAs with automatic placement and routing tools. The static time analysis results suggests that the proposed network implementation could generate 2.7 billion connections per second performance

An investigation into adaptive power reduction techniques for neural hardware

In light of the growing applicability of Artificial Neural Network (ANN) in the signal processing field [1] and the present thrust of the semiconductor industry towards lowpower SOCs for mobile devices [2], the power consumption of ANN hardware has become a very important implementation issue. Adaptability is a powerful and useful feature of neural networks. All current approaches for low-power ANN hardware techniques are ‘non-adaptive’ with respect to the power consumption of the network (i.e. power-reduction is not an objective of the adaptation/learning process). In the research work presented in this thesis, investigations on possible adaptive power reduction techniques have been carried out, which attempt to exploit the adaptability of neural networks in order to reduce the power consumption. Three separate approaches for such adaptive power reduction are proposed: adaptation of size, adaptation of network weights and adaptation of calculation precision. Initial case studies exhibit promising results with significantpower reduction

JPEG-like Image Compression using Neural-network-based Block Classification and Adaptive Reordering of Transform Coefficients

The research described in this thesis addresses aspects of coding of discrete-cosinetransform (DCT) coefficients, that are present in a variety of transform-based digital-image-compression schemes such as JPEG. Coefficient reordering; that directly affects the symbol statistics for entropy coding, and therefore the effectiveness of entropy coding; is investigated. Adaptive zigzag reordering, a novel versatile technique that achieves efficient reordering by processing variable-size rectangular sub-blocks of coefficients, is developed. Classification of blocks of DCT coefficients using an artificial neural network (ANN) prior to adaptive zigzag reordering is also considered. Some established digital-image-compression techniques are reviewed, and the JPEG standard for the DCT-based method is studied in more detail. An introduction to artificial neural networks is provided. Lossless conversion of blocks of coefficients using adaptive zigzag reordering is investigated, and experimental results are presented. A versatile algorithm, that generates zigzag scan paths for sub-blocks of any dimensions using a binary decision tree, is developed. An implementation of the algorithm based on programmable logic devices (PLDs) is described demonstrating the feasibility of hardware implementations. Coding of the sub-block dimensions, that need to be retained in order to reconstruct a sub-block during decoding, based on the scan-path length is developed. Lossy conversion of blocks of coefficients is also considered, and experimental results are presented. A two-layer feedforward artificial neural network trained using an error-backpropagation algorithm, that determines the sub-block dimensions, is described. Isolated nonzero coefficients of small significance are discarded in some blocks, and therefore smaller sub-blocks are generated

Inter-domain routing: pricing policy and route selection using neural networks.

by Wong Leung-Chung Chris.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.Includes bibliographical references (leaves 86-[92]).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Routing Overview --- p.2Chapter 1.2 --- Routing in the Internet --- p.5Chapter 1.2.1 --- Inter-Domain Routing --- p.6Chapter 1.2.2 --- Intra-Domain Routing --- p.7Chapter 1.2.3 --- The Future Trend --- p.7Chapter 2 --- Inter-Domain Routing --- p.9Chapter 2.1 --- Inter-Domain Routing Protocols --- p.9Chapter 2.1.1 --- Exterior Gateway Protocol (EGP) --- p.10Chapter 2.1.2 --- Border Gateway Protocol (BGP) --- p.11Chapter 2.1.3 --- Inter-Domain Policy Routing (IDPR) --- p.12Chapter 2.1.4 --- Other Protocols --- p.13Chapter 2.2 --- The Need for Pricing on Inter-Domain Routing Protocols --- p.13Chapter 2.3 --- Pricing Scheme on the Inter-Domain level --- p.15Chapter 2.4 --- Routing Protocols to Support Pricing on the Internet --- p.16Chapter 2.4.1 --- Routing Towards Multiple-Additive Metrics --- p.16Chapter 2.4.2 --- "Network Model, Notations and Assumptions" --- p.16Chapter 2.4.3 --- The Problem Statement --- p.18Chapter 3 --- Application of Neural Nets in Route Selection --- p.20Chapter 3.1 --- Neural Network (NN) Overview --- p.20Chapter 3.1.1 --- Brief History on Neural Network Research --- p.20Chapter 3.1.2 --- Definition of Neural Network --- p.21Chapter 3.1.3 --- Neural Network Architectures --- p.22Chapter 3.1.4 --- Transfer Fucntion of a Neuron --- p.24Chapter 3.1.5 --- Learning Methods --- p.25Chapter 3.1.6 --- Applications in Telecommunications --- p.26Chapter 3.2 --- Review on the Applications of Neural Networks in Packet Routing --- p.27Chapter 3.2.1 --- The JEB Branch --- p.27Chapter 3.2.2 --- The Hopfield/Energy Minimization Branch (HEM) --- p.29Chapter 3.2.3 --- Supervised Learning (SL) --- p.34Chapter 3.3 --- Discussions --- p.35Chapter 4 --- Route Selection as “Link-state´ح Classification --- p.36Chapter 4.1 --- Multi-Layer Feedforward Network (MLFN) --- p.37Chapter 4.1.1 --- Function Approximation Power of MLFN --- p.38Chapter 4.1.2 --- Choosing MLFN parameters..........´ب --- p.40Chapter 4.1.3 --- Trailing a MLFN --- p.41Chapter 4.2 --- The Utility Function --- p.43Chapter 4.3 --- The Neural Network Architecture --- p.46Chapter 4.3.1 --- Routing Graph Representation with Successor Sequence Table (SST) --- p.46Chapter 4.3.2 --- The Neural Network Layout --- p.52Chapter 4.3.3 --- How the Neural Network Controller Works --- p.55Chapter 4.3.4 --- Training --- p.56Chapter 4.4 --- Simulation --- p.56Chapter 4.4.1 --- Performance Parameters --- p.56Chapter 4.4.2 --- Simulation Results --- p.57Chapter 4.5 --- Conclusions and Discussions --- p.70Chapter 5 --- Route Selection as Energy Minimization - A Theoretical Study --- p.73Chapter 5.1 --- The Hopfield/Tank NN Model --- p.73Chapter 5.2 --- Boltzman's Machine --- p.76Chapter 5.3 --- Boltzman's Machine Model for Multiple-Metrices Routing --- p.79Chapter 5.4 --- Conclusions --- p.82Chapter 6 --- Conclusions --- p.84Bibliography --- p.8

Штучний інтелект

Funding: Research, preparation of materials and preparation of the textbook were carried out under the project – grant no. PPI/KAT/2019/1/00015/U/00001 "Cognitive technologies – second-cycle studies in English" and were carried under the KATAMARAN program Polish National Agency for Academic Exchange (NAWA). The program is co-financed by the European Social Fund under the Knowledge Education Development Operational Program, a non-competition project entitled "Supporting the institutional capacity of Polish universities through the creation and implementation of international study programs" implemented under Measure 3.3. Internationalization of Polish higher education, specified in the application for project funding no. POWR.03.03.00-00-PN 16/18. The project was carried out in cooperation with the Silesian University of Technology (project leader – Poland) and the Kiev National University of Construction and Architecture (project partner – Ukraine).Фінансування: Дослідження, підготовка матеріалів та підготовка підручника були здійснені в рамках проекту - грант №. PPI/KAT/2019/1/00015/U/00001 "Когнітивні технології-навчання другого циклу англійською мовою", які здійснювалися за програмою КАТАМАРАН Польське національне агентство академічного обміну (NAWA) . Програма спільно фінансується Європейським соціальним фондом у рамках програми "Знання" Оперативна програма розвитку освіти, позаконкурентний проект під назвою "Підтримка інституційної спроможності польських університетів через створення та реалізація міжнародних навчальних програм ", що реалізуються відповідно до Заходу 3.3. Інтернаціоналізація польської вищої освіти, зазначена у заявці на фінансування проекту POWR.03.03.00-00-PN 16/18. Проект здійснювався у співпраці з Сілезьким технологічним університетом (керівник проекту - Польща) та Київським національним університетом будівництва та архітектури (партнер проекту - Україна)

Efficient Neuromorphic Computing Enabled by Spin-Transfer Torque: Devices, Circuits and Systems

Present day computers expend orders of magnitude more computational resources to perform various cognitive and perception related tasks that humans routinely perform everyday. This has recently resulted in a seismic shift in the field of computation where research efforts are being directed to develop a neurocomputer that attempts to mimic the human brain by nanoelectronic components and thereby harness its efficiency in recognition problems. Bridging the gap between neuroscience and nanoelectronics, this thesis demonstrates the encoding of biological neural and synaptic functionalities in the underlying physics of electron spin. Description of various spin-transfer torque mechanisms that can be potentially utilized for realizing neuro-mimetic device structures is provided. A cross-layer perspective extending from the device to the circuit and system level is presented to envision the design of an All-Spin neuromorphic processor enabled with on-chip learning functionalities. Device-circuit-algorithm co-simulation framework calibrated to experimental results suggest that such All-Spin neuromorphic systems can potentially achieve almost two orders of magnitude energy improvement in comparison to state-of-the-art CMOS implementations

Object-oriented analysis and design of computational intelligence systems

Machine learning from data, neuro-fuzzy information processing, approximate reasoning and genetic and evolutionary computation are all aspects of computational intelligence (also called soft computing methods). Soft computing methods differ from conventional computing in that they are tolerant of imprecision, uncertainty and partial truths. These characteristics can be exploited to achieve tractability, robustness and low solution costs when the solution to a complex (in machine terms) problem is required. The principal constituents of soft computing include: Neural Networks, Fuzzy Logic and Probabilistic Reasoning Systems. Genetic Algorithms (GAs), Evolutionary Algorithms, Chaos Theory', Complexity Theory and parts of Learning Theory all come under Probabilistic Reasoning Systems. Hybrid systems can be designed incorporating 2 or more aspects of soft computing that are more powerful than any of the components used in a stand alone fashion. A unified framework is needed to implement and manipulate such systems. Such a framework will allow for easy visualisation of the underlying concepts and easy modification of the resulting computer models. In this thesis, an investigation of the major aspects of computational intelligence has been carried out. The main emphasis has been placed on developing an object-oriented framework for architecting computational intelligence systems. Object models for Neural Networks, Fuzzy Logic Systems and Evolutionary Computation systems have been developed. Software has been written in C++ to realise sample implementations of the various systems. Finally, practical applications and the results of using the Neural Networks, Fuzzy Logic systems and Genetic Algorithms developed in solving real world problems are presented. A consistent notation based on the Object Modelling Technique (OMT) is used throughout the thesis to describe the software architectures from which the computer implementation models have been derived