Strategies for neural networks in ballistocardiography with a view towards hardware implementation

A thesis submitted for the degree of Doctor of Philosophy at the University of LutonThe work described in this thesis is based on the results of a clinical trial conducted by the research team at the Medical Informatics Unit of the University of Cambridge, which show that the Ballistocardiogram (BCG) has prognostic value in detecting impaired left ventricular function before it becomes clinically overt as myocardial infarction leading to sudden death. The objective of this study is to develop and demonstrate a framework for realising an on-line BCG signal classification model in a portable device that would have the potential to find pathological signs as early as possible for home health care. Two new on-line automatic BeG classification models for time domain BeG classification are proposed. Both systems are based on a two stage process: input feature extraction followed by a neural classifier. One system uses a principal component analysis neural network, and the other a discrete wavelet transform, to reduce the input dimensionality. Results of the classification, dimensionality reduction, and comparison are presented. It is indicated that the combined wavelet transform and MLP system has a more reliable performance than the combined neural networks system, in situations where the data available to determine the network parameters is limited. Moreover, the wavelet transfonn requires no prior knowledge of the statistical distribution of data samples and the computation complexity and training time are reduced. Overall, a methodology for realising an automatic BeG classification system for a portable instrument is presented. A fully paralJel neural network design for a low cost platform using field programmable gate arrays (Xilinx's XC4000 series) is explored. This addresses the potential speed requirements in the biomedical signal processing field. It also demonstrates a flexible hardware design approach so that an instrument's parameters can be updated as data expands with time. To reduce the hardware design complexity and to increase the system performance, a hybrid learning algorithm using random optimisation and the backpropagation rule is developed to achieve an efficient weight update mechanism in low weight precision learning. The simulation results show that the hybrid learning algorithm is effective in solving the network paralysis problem and the convergence is much faster than by the standard backpropagation rule. The hidden and output layer nodes have been mapped on Xilinx FPGAs with automatic placement and routing tools. The static time analysis results suggests that the proposed network implementation could generate 2.7 billion connections per second performance

Approches neuronales pour l'extraction des composantes principales d'images multispectrales de télédétection

Le problème traité dans le présent article consiste en l'extraction des composantes principales les plus significatives d'images multispectrales de télédétection sans avoir à calculer la matrice de covariance des images spectrales. L'originalité du travail réside dans l'élaboration des algorithmes d'apprentissage spécifiques pour deux approches neuronales d'Analyse en Composantes Principales (ACP). Les deux approches possèdent des convergences rapides. L'application sur une image multispectrale réelle a montré leur efficacité dans l'extraction des composantes principales les plus significatives. The problem addressed in the présent paper is the most significant principal components extraction of remotely sensed multispectral images without having to calculate the covariance matrix of spectral images. The originality of the work resides in the elaboration of specific training algorithms for two neural network-based approaches of Principal Component Analysis (PCA). The convergence of a proposed approaches are rapid. The application on a real multispectral image has shown their efficiency in the extraction of the most significant principal components

Sensor encoding using lateral inhibited, self-organized cellular neural networks

The paper focuses on the division of the sensor field into subsets of sensor events and proposes the linear transformation with the smallest achievable error for reproduction: the transform coding approach using the principal component analysis (PCA). For the implementation of the PCA, this paper introduces a new symmetrical, lateral inhibited neural network model, proposes an objective function for it and deduces the corresponding learning rules. The necessary conditions for the learning rate and the inhibition parameter for balancing the crosscorrelations vs. the autocorrelations are computed. The simulation reveals that an increasing inhibition can speed up the convergence process in the beginning slightly. In the remaining paper, the application of the network in picture encoding is discussed. Here, the use of non-completely connected networks for the self-organized formation of templates in cellular neural networks is shown. It turns out that the self-organizing Kohonen map is just the non-linear, first order approximation of a general self-organizing scheme. Hereby, the classical transform picture coding is changed to a parallel, local model of linear transformation by locally changing sets of self-organized eigenvector projections with overlapping input receptive fields. This approach favors an effective, cheap implementation of sensor encoding directly on the sensor chip. Keywords: Transform coding, Principal component analysis, Lateral inhibited network, Cellular neural network, Kohonen map, Self-organized eigenvector jets

A Generalized Neural Network Approach to Mobile Robot Navigation and Obstacle Avoidance

In this thesis, we tackle the problem of extending neural network navigation algorithms for various types of mobile robots and 2-dimensional range sensors. We propose a general method to interpret the data from various types of 2-dimensional range sensors and a neural network algorithm to perform the navigation task. Our approach can yield a global navigation algorithm which can be applied to various types of range sensors and mobile robot platforms. Moreover, this method allows the neural networks to be trained using only one type of 2-dimensional range sensor, which contributes positively to reducing the time required for training the networks. Experimental results carried out in simulation environments demonstrate the effectiveness of our approach in mobile robot navigation for different kinds of robots and sensors. Therefore, the successful implementation of our method provides a solution to apply mobile robot navigation algorithms to various robot platforms

Beta hebbian learning: definition and analysis of a new family of learning rules for exploratory projection pursuit

[EN] This thesis comprises an investigation into the derivation of learning rules in artificial neural networks from probabilistic criteria. •Beta Hebbian Learning (BHL). First of all, it is derived a new family of learning rules which are based on maximising the likelihood of the residual from a negative feedback network when such residual is deemed to come from the Beta Distribution, obtaining an algorithm called Beta Hebbian Learning, which outperforms current neural algorithms in Exploratory Projection Pursuit. • Beta-Scale Invariant Map (Beta-SIM). Secondly, Beta Hebbian Learning is applied to a well-known Topology Preserving Map algorithm called Scale Invariant Map (SIM) to design a new of its version called Beta-Scale Invariant Map (Beta-SIM). It is developed to facilitate the clustering and visualization of the internal structure of high dimensional complex datasets effectively and efficiently, specially those characterized by having internal radial distribution. The Beta-SIM behaviour is thoroughly analysed comparing its results, in terms performance quality measures with other well-known topology preserving models. • Weighted Voting Superposition Beta-Scale Invariant Map (WeVoS-Beta-SIM). Finally, the use of ensembles such as the Weighted Voting Superposition (WeVoS) is tested over the previous novel Beta-SIM algorithm, in order to improve its stability and to generate accurate topology maps when using complex datasets. Therefore, the WeVoS-Beta-Scale Invariant Map (WeVoS-Beta-SIM), is presented, analysed and compared with other well-known topology preserving models. All algorithms have been successfully tested using different artificial datasets to corroborate their properties and also with high-complex real datasets.[ES] Esta tesis abarca la investigación sobre la derivación de reglas de aprendizaje en redes neuronales artificiales a partir de criterios probabilísticos. • Beta Hebbian Learning (BHL). En primer lugar, se deriva una nueva familia de reglas de aprendizaje basadas en maximizar la probabilidad del residuo de una red con retroalimentación negativa cuando se considera que dicho residuo proviene de la Distribución Beta, obteniendo un algoritmo llamado Beta Hebbian Learning, que mejora a algoritmos neuronales actuales de búsqueda de proyecciones exploratorias. • Beta-Scale Invariant Map (Beta-SIM). En Segundo lugar, Beta Hebbian Learning se aplica a un conocido algoritmo de Mapa de Preservación de la Topología llamado Scale Invariant Map (SIM) para diseñar una nueva versión llamada Beta-Scale Invariant Map (Beta-SIM). Este nuevo algoritmo ha sido desarrollado para facilitar el agrupamiento y visualización de la estructura interna de conjuntos de datos complejos de alta dimensionalidad de manera eficaz y eficiente, especialmente aquellos caracterizados por tener una distribución radial interna. El comportamiento de Beta-SIM es analizado en profundidad comparando sus resultados, en términos de medidas de calidad de rendimiento con otros modelos bien conocidos de preservación de topología. • Weighted Voting Superposition Beta-Scale Invariant Map (WeVoS-Beta-SIM). Finalmente, el uso de ensembles como el Weighted Voting Superposition (WeVoS) sobre el algoritmo Beta-SIM es probado, con objeto de mejorar su estabilidad y generar mapas topológicos precisos cuando se utilizan conjuntos de datos complejos. Por lo tanto, se presenta, analiza y compara el WeVoS-Beta-Scale Invariant Map (WeVoS-Beta-SIM) con otros modelos bien conocidos de preservación de topología. Todos los algoritmos han sido probados con éxito sobre conjuntos de datos artificiales para corroborar sus propiedades, así como con conjuntos de datos reales de gran complejidad

An investigation of the design and use of feed-forward artificial neural networks in the classification of remotely sensed images

Artificial neural networks (ANNs) have attracted the attention of researchers in many fields, and have been used to solve a wide range of problems. In the field of remote sensing they have been used in a variety of applications, including land cover mapping, image compression, geological mapping and meteorological image classification, and have generally proved to be more powerful than conventional statistical classifiers, especially when training data are limited and the data in each class are not normally distributed. The use of ANNs requires some critical decisions on the part of the user. These decisions, which are mainly concerned with the determinations of the components of the network structure and the parameters defined for the learning algorithm, can significantly affect the accuracy of the resulting classification. Although there are some discussions in the literature regarding the issues that affect network performance, there is no standard method or approach that is universally accepted to determine the optimum values of these parameters for a particular problem. In this thesis, a feed-forward network structure that learns the characteristics of the training data through the backpropagation learning algorithm is employed to classify land cover features using multispectral, multitemporal, and multisensory image data. The thesis starts with a review and discussion of general principles of classification and the use of artificial neural networks. Special emphasis is put on the issue of feature selection, due to the availability of hyperspectral image data from recent sensors. The primary aims of this research are to comprehensively investigate the impact of the choice of network architecture and initial parameter estimates, and to compare a number of heuristics developed by researchers. The most effective heuristics are identified on the basis of a large number of experiments employing two real-world datasets, and the superiority of the optimum settings using the 'best' heuristics is then validated using an independent dataset. The results are found to be promising in terms of ease of design and use of ANNs, and in producing considerably higher classification accuracies than either the maximum likelihood or neural network classifiers constructed using ad hoc design and implementation strategies. A number of conclusions are drawn and later used to generate a comprehensive set of guidelines that will facilitate the process of design and use of artificial neural networks in remote sensing image classification. This study also explores the use of visualisation techniques in understanding the behaviour of artificial neural networks and the results produced by them. A number of visual analysis techniques are employed to examine the internal characteristics of the training data. For this purpose, a toolkit allowing the analyst to perform a variety of visualisation and analysis procedures was created using the MATLAB software package, and is available in the accompanying CD-ROM. This package was developed during the course of this research, and contains the tools used during the investigations reported in this thesis. The contribution to knowledge of the research work reported in this thesis lies in the identification of optimal strategies for the use of ANNs in land cover classifications based on remotely sensed data. Further contributions include an indepth analysis of feature selection methods for use with high-dimensional datasets, and the production of a MATLAB toolkit that implements the methods used in this study

Biologically inspired feature extraction for rotation and scale tolerant pattern analysis

Biologically motivated information processing has been an important area of scientific research for decades. The central topic addressed in this dissertation is utilization of lateral inhibition and more generally, linear networks with recurrent connectivity along with complex-log conformal mapping in machine based implementations of information encoding, feature extraction and pattern recognition. The reasoning behind and method for spatially uniform implementation of inhibitory/excitatory network model in the framework of non-uniform log-polar transform is presented. For the space invariant connectivity model characterized by Topelitz-Block-Toeplitz matrix, the overall network response is obtained without matrix inverse operations providing the connection matrix generating function is bound by unity. It was shown that for the network with the inter-neuron connection function expandable in a Fourier series in polar angle, the overall network response is steerable. The decorrelating/whitening characteristics of networks with lateral inhibition are used in order to develop space invariant pre-whitening kernels specialized for specific category of input signals. These filters have extremely small memory footprint and are successfully utilized in order to improve performance of adaptive neural whitening algorithms. Finally, the method for feature extraction based on localized Independent Component Analysis (ICA) transform in log-polar domain and aided by previously developed pre-whitening filters is implemented. Since output codes produced by ICA are very sparse, a small number of non-zero coefficients was sufficient to encode input data and obtain reliable pattern recognition performance

An investigation of the design and use of feed-forward artificial neural networks in the classification of remotely sensed images

Artificial neural networks (ANNs) have attracted the attention of researchers in many fields, and have been used to solve a wide range of problems. In the field of remote sensing they have been used in a variety of applications, including land cover mapping, image compression, geological mapping and meteorological image classification, and have generally proved to be more powerful than conventional statistical classifiers, especially when training data are limited and the data in each class are not normally distributed. The use of ANNs requires some critical decisions on the part of the user. These decisions, which are mainly concerned with the determinations of the components of the network structure and the parameters defined for the learning algorithm, can significantly affect the accuracy of the resulting classification. Although there are some discussions in the literature regarding the issues that affect network performance, there is no standard method or approach that is universally accepted to determine the optimum values of these parameters for a particular problem. In this thesis, a feed-forward network structure that learns the characteristics of the training data through the backpropagation learning algorithm is employed to classify land cover features using multispectral, multitemporal, and multisensory image data. The thesis starts with a review and discussion of general principles of classification and the use of artificial neural networks. Special emphasis is put on the issue of feature selection, due to the availability of hyperspectral image data from recent sensors. The primary aims of this research are to comprehensively investigate the impact of the choice of network architecture and initial parameter estimates, and to compare a number of heuristics developed by researchers. The most effective heuristics are identified on the basis of a large number of experiments employing two real-world datasets, and the superiority of the optimum settings using the 'best' heuristics is then validated using an independent dataset. The results are found to be promising in terms of ease of design and use of ANNs, and in producing considerably higher classification accuracies than either the maximum likelihood or neural network classifiers constructed using ad hoc design and implementation strategies. A number of conclusions are drawn and later used to generate a comprehensive set of guidelines that will facilitate the process of design and use of artificial neural networks in remote sensing image classification. This study also explores the use of visualisation techniques in understanding the behaviour of artificial neural networks and the results produced by them. A number of visual analysis techniques are employed to examine the internal characteristics of the training data. For this purpose, a toolkit allowing the analyst to perform a variety of visualisation and analysis procedures was created using the MATLAB software package, and is available in the accompanying CD-ROM. This package was developed during the course of this research, and contains the tools used during the investigations reported in this thesis. The contribution to knowledge of the research work reported in this thesis lies in the identification of optimal strategies for the use of ANNs in land cover classifications based on remotely sensed data. Further contributions include an indepth analysis of feature selection methods for use with high-dimensional datasets, and the production of a MATLAB toolkit that implements the methods used in this study