Optimization Techniques Applied to Neural Networks

Electrical Engineerin

An optimal nephelometric model design method for particle characterisation

Scattering nephelometry is a particle characterisation method applicable to fluid suspensions containing impurities. Solutions derived by the method feature particle classification by size (diameter), volume or texture as well as continuous on-line and in-situ monitoring, The replacement of turbidimeters with nephelometers in many existing turbidity applications could result in suppression of side effects caused by limitations and uncontrolled parameter drifts and satisfaction of problem-defined constraints at virtually no change in implementation cost. A major issue of nephelometric model design is the selection of a mathematical tool suitable for the modelling of the data analysis system. [Continues.

Photovoltaic power forecast modeling with artificial neural networks

Dissertação de Mestrado, Engenharia Eletrónica e Telecomunicações, Faculdade de Ciências e Tecnologia, Universidade do Algarve, 2016Com uma crescente preocupação relativamente ao consumo energético global, a energia fotovoltaica surge como uma fonte energia renovável promissora. Esta dissertação é constru ída sob a premissa de que a capacidade de previsão de potência fotovoltaica produzida possibilita o aumento de performance da rede elétrica local através de um controlo eficiente da mesma. O trabalho desenvolvido propõe uma estrutura com a capacidade de previsão de potência produzida por um sistema fotovoltaico ligado a rede elétrica presente na Universidade do Algarve. A estrutura de previsão proposta é composta por dois modelos dinâmicos, não lineares, de previsão e um modelo estático não linear. Redes Neuronais Artificiais foram usadas como modelos. Os modelos de previsão têm como objectivo fazer previsões da temperatura do ar e irradiação solar em passos incrementais de 5 minutos para um horizonte de previsão de 4 horas. O modelo estático é construído para estimar a potência gerada pelo sistema fotovoltaico e é otimizado através de comparação entre vários tipos de redes neuronais como o perceptrão multicamadas e funções de base radial, e modelos com escalas temporais diferentes, aplicados a diferentes estações do ano, bem como um modelo anual.In a growing concern for the world energy consumption, photovoltaic energy sources are a reliable renewable energy alternative. This thesis is built upon the premise that the forecast of photovoltaic power production can increase performance of local electric network through an efficient network management. The work developed proposes a power production forecast structure based on a grid-connected photovoltaic system in the University of Algarve. The proposed forecast structure is composed of two non-linear dynamic forecasting models and one non-linear static model. Artificial Neural Networks were used in the development of these models which are intended to forecast solar irradiance and air temperature using Radial Basis Functions with 5 minutes time steps within a prediction horizon of 4 hours. The static model on the structure was created to estimate the power generated by the photovoltaic system and it was optimized through comparison between several network architectures (MLP and RBF) and several seasonal models, as well as a annual model

A generalised feedforward neural network architecture and its applications to classification and regression

Shunting inhibition is a powerful computational mechanism that plays an important role in sensory neural information processing systems. It has been extensively used to model some important visual and cognitive functions. It equips neurons with a gain control mechanism that allows them to operate as adaptive non-linear filters. Shunting Inhibitory Artificial Neural Networks (SIANNs) are biologically inspired networks where the basic synaptic computations are based on shunting inhibition. SIANNs were designed to solve difficult machine learning problems by exploiting the inherent non-linearity mediated by shunting inhibition. The aim was to develop powerful, trainable networks, with non-linear decision surfaces, for classification and non-linear regression tasks. This work enhances and extends the original SIANN architecture to a more general form called the Generalised Feedforward Neural Network (GFNN) architecture, which contains as subsets both SIANN and the conventional Multilayer Perceptron (MLP) architectures. The original SIANN structure has the number of shunting neurons in the hidden layers equal to the number of inputs, due to the neuron model that is used having a single direct excitatory input. This was found to be too restrictive, often resulting in inadequately small or inordinately large network structures

Trajectory prediction of moving objects by means of neural networks

Thesis (Master)--Izmir Institute of Technology, Computer Engineering, Izmir, 1997Includes bibliographical references (leaves: 103-105)Text in English; Abstract: Turkish and Englishviii, 105 leavesEstimating the three-dimensional motion of an object from a sequence of object positions and orientation is of significant importance in variety of applications in control and robotics. For instance, autonomous navigation, manipulation, servo, tracking, planning and surveillance needs prediction of motion parameters. Although "motion estimation" is an old problem (the formulations date back to the beginning of the century), only recently scientists have provided with the tools from nonlinear system estimation theory to solve this problem eural Networks are the ones which have recently been used in many nonlinear dynamic system parameter estimation context. The approximating ability of the neural network is used to identifY the relation between system variables and parameters of a dynamic system. The position, velocity and acceleration of the object are estimated by several neural networks using the II most recent measurements of the object coordinates as input to the system Several neural network topologies with different configurations are introduced and utilized in the solution of the problem. Training schemes for each configuration are given in detail. Simulation results for prediction of motion having different characteristics via different architectures with alternative configurations are presented comparatively

Identifying Humans by the Shape of Their Heartbeats and Materials by Their X-Ray Scattering Profiles

Security needs at access control points presents itself in the form of human identification and/or material identification. The field of Biometrics deals with the problem of identifying individuals based on the signal measured from them. One approach to material identification involves matching their x-ray scattering profiles with a database of known materials. Classical biometric traits such as fingerprints, facial images, speech, iris and retinal scans are plagued by potential circumvention they could be copied and later used by an impostor. To address this problem, other bodily traits such as the electrical signal acquired from the brain (electroencephalogram) or the heart (electrocardiogram) and the mechanical signals acquired from the heart (heart sound, laser Doppler vibrometry measures of the carotid pulse) have been investigated. These signals depend on the physiology of the body, and require the individual to be alive and present during acquisition, potentially overcoming circumvention. We investigate the use of the electrocardiogram (ECG) and carotid laser Doppler vibrometry (LDV) signal, both individually and in unison, for biometric identity recognition. A parametric modeling approach to system design is employed, where the system parameters are estimated from training data. The estimated model is then validated using testing data. A typical identity recognition system can operate in either the authentication (verification) or identification mode. The performance of the biometric identity recognition systems is evaluated using receiver operating characteristic (ROC) or detection error tradeoff (DET) curves, in the authentication mode, and cumulative match characteristic (CMC) curves, in the identification mode. The performance of the ECG- and LDV-based identity recognition systems is comparable, but is worse than those of classical biometric systems. Authentication performance below 1% equal error rate (EER) can be attained when the training and testing data are obtained from a single measurement session. When the training and testing data are obtained from different measurement sessions, allowing for a potential short-term or long-term change in the physiology, the authentication EER performance degrades to about 6 to 7%. Leveraging both the electrical (ECG) and mechanical (LDV) aspects of the heart, we obtain a performance gain of over 50%, relative to each individual ECG-based or LDV-based identity recognition system, bringing us closer to the performance of classical biometrics, with the added advantage of anti-circumvention. We consider the problem of designing combined x-ray attenuation and scatter systems and the algorithms to reconstruct images from the systems. As is the case within a computational imaging framework, we tackle the problem by taking a joint system and algorithm design approach. Accurate modeling of the attenuation of incident and scattered photons within a scatter imaging setup will ultimately lead to more accurate estimates of the scatter densities of an illuminated object. Such scattering densities can then be used in material classification. In x-ray scatter imaging, tomographic measurements of the forward scatter distribution are used to infer scatter densities within a volume. A mask placed between the object and the detector array provides information about scatter angles. An efficient computational implementation of the forward and backward model facilitates iterative algorithms based upon a Poisson log-likelihood. The design of the scatter imaging system influences the algorithmic choices we make. In turn, the need for efficient algorithms guides the system design. We begin by analyzing an x-ray scatter system fitted with a fanbeam source distribution and flat-panel energy-integrating detectors. Efficient algorithms for reconstructing object scatter densities from scatter measurements made on this system are developed. Building on the fanbeam source, energy-integrating at-panel detection model, we develop a pencil beam model and an energy-sensitive detection model. The scatter forward models and reconstruction algorithms are validated on simulated, Monte Carlo, and real data. We describe a prototype x-ray attenuation scanner, co-registered with the scatter system, which was built to provide complementary attenuation information to the scatter reconstruction and present results of applying alternating minimization reconstruction algorithms on measurements from the scanner

Application of geotechnical monitoring in tunnels with neural networks and finite elements methods

Εθνικό Μετσόβιο Πολυτεχνείο--Μεταπτυχιακή Εργασία. Διεπιστημονικό-Διατμηματικό Πρόγραμμα Μεταπτυχιακών Σπουδών (Δ.Π.Μ.Σ.) “Σχεδιασμός και Κατασκευή Υπόγειων Έργων

Improving time efficiency of feedforward neural network learning

Feedforward neural networks have been widely studied and used in many applications in science and engineering. The training of this type of networks is mainly undertaken using the well-known backpropagation based learning algorithms. One major problem with this type of algorithms is the slow training convergence speed, which hinders their applications. In order to improve the training convergence speed of this type of algorithms, many researchers have developed different improvements and enhancements. However, the slow convergence problem has not been fully addressed. This thesis makes several contributions by proposing new backpropagation learning algorithms based on the terminal attractor concept to improve the existing backpropagation learning algorithms such as the gradient descent and Levenberg-Marquardt algorithms. These new algorithms enable fast convergence both at a distance from and in a close range of the ideal weights. In particular, a new fast convergence mechanism is proposed which is based on the fast terminal attractor concept. Comprehensive simulation studies are undertaken to demonstrate the effectiveness of the proposed backpropagataion algorithms with terminal attractors. Finally, three practical application cases of time series forecasting, character recognition and image interpolation are chosen to show the practicality and usefulness of the proposed learning algorithms with comprehensive comparative studies with existing algorithms