Generalised correlation higher order neural networks, neural network operation and Levenberg-Marquardt training on field programmable gate arrays

Higher Order Neural Networks (HONNs) were introduced in the late 80's as a solution to the increasing complexity within Neural Networks (NNs). Similar to NNs HONNs excel at performing pattern recognition, classification, optimisation particularly for non-linear systems in varied applications such as communication channel equalisation, real time intelligent control, and intrusion detection. This research introduced new HONNs called the Generalised Correlation Higher Order Neural Networks which as an extension to the ordinary first order NNs and HONNs, based on interlinked arrays of correlators with known relationships, they provide the NN with a more extensive view by introducing interactions between the data as an input to the NN model. All studies included two data sets to generalise the applicability of the findings. The research investigated the performance of HONNs in the estimation of short term returns of two financial data sets, the FTSE 100 and NASDAQ. The new models were compared against several financial models and ordinary NNs. Two new HONNs, the Correlation HONN (C-HONN) and the Horizontal HONN (Horiz-HONN) outperformed all other models tested in terms of the Akaike Information Criterion (AIC). The new work also investigated HONNs for camera calibration and image mapping. HONNs were compared against NNs and standard analytical methods in terms of mapping performance for three cases; 3D-to-2D mapping, a hybrid model combining HONNs with an analytical model, and 2D-to-3D inverse mapping. This study considered 2 types of data, planar data and co-planar (cube) data. To our knowledge this is the first study comparing HONNs against NNs and analytical models for camera calibration. HONNs were able to transform the reference grid onto the correct camera coordinate and vice versa, an aspect that the standard analytical model fails to perform with the type of data used. HONN 3D-to-2D mapping had calibration error lower than the parametric model by up to 24% for plane data and 43% for cube data. The hybrid model also had lower calibration error than the parametric model by 12% for plane data and 34% for cube data. However, the hybrid model did not outperform the fully non-parametric models. Using HONNs for inverse mapping from 2D-to-3D outperformed NNs by up to 47% in the case of cube data mapping. This thesis is also concerned with the operation and training of NNs in limited precision specifically on Field Programmable Gate Arrays (FPGAs). Our findings demonstrate the feasibility of on-line, real-time, low-latency training on limited precision electronic hardware such as Digital Signal Processors (DSPs) and FPGAs. This thesis also investigated the e�ffects of limited precision on the Back Propagation (BP) and Levenberg-Marquardt (LM) optimisation algorithms. Two new HONNs are compared against NNs for estimating the discrete XOR function and an optical waveguide sidewall roughness dataset in order to find the Minimum Precision for Lowest Error (MPLE) at which the training and operation are still possible. The new findings show that compared to NNs, HONNs require more precision to reach a similar performance level, and that the 2nd order LM algorithm requires at least 24 bits of precision. The final investigation implemented and demonstrated the LM algorithm on Field Programmable Gate Arrays (FPGAs) for the first time in our knowledge. It was used to train a Neural Network, and the estimation of camera calibration parameters. The LM algorithm approximated NN to model the XOR function in only 13 iterations from zero initial conditions with a speed-up in excess of 3 x 10^6 compared to an implementation in software. Camera calibration was also demonstrated on FPGAs; compared to the software implementation, the FPGA implementation led to an increase in the mean squared error and standard deviation of only 17.94% and 8.04% respectively, but the FPGA increased the calibration speed by a factor of 1:41 x 106

Engineering Education and Research Using MATLAB

MATLAB is a software package used primarily in the field of engineering for signal processing, numerical data analysis, modeling, programming, simulation, and computer graphic visualization. In the last few years, it has become widely accepted as an efficient tool, and, therefore, its use has significantly increased in scientific communities and academic institutions. This book consists of 20 chapters presenting research works using MATLAB tools. Chapters include techniques for programming and developing Graphical User Interfaces (GUIs), dynamic systems, electric machines, signal and image processing, power electronics, mixed signal circuits, genetic programming, digital watermarking, control systems, time-series regression modeling, and artificial neural networks

Applications of MATLAB in Science and Engineering

The book consists of 24 chapters illustrating a wide range of areas where MATLAB tools are applied. These areas include mathematics, physics, chemistry and chemical engineering, mechanical engineering, biological (molecular biology) and medical sciences, communication and control systems, digital signal, image and video processing, system modeling and simulation. Many interesting problems have been included throughout the book, and its contents will be beneficial for students and professionals in wide areas of interest

Rapid Digital Architecture Design of Computationally Complex Algorithms

Traditional digital design techniques hardly keep up with the rising abundance of programmable circuitry found on recent Field-Programmable Gate Arrays. Therefore, the novel Rapid Data Type-Agnostic Digital Design Methodology (RDAM) elevates the design perspective of digital design engineers away from the register-transfer level to the algorithmic level. It is founded on the capabilities of High-Level Synthesis tools. By consequently working with data type-agnostic source codes, the RDAM brings significant simplifications to the fixed-point conversion of algorithms and the design of complex-valued architectures. Signal processing applications from the field of Compressed Sensing illustrate the efficacy of the RDAM in the context of multi-user wireless communications. For instance, a complex-valued digital architecture of Orthogonal Matching Pursuit with rank-1 updating has successfully been implemented and tested

Advancements in Real-Time Simulation of Power and Energy Systems

Modern power and energy systems are characterized by the wide integration of distributed generation, storage and electric vehicles, adoption of ICT solutions, and interconnection of different energy carriers and consumer engagement, posing new challenges and creating new opportunities. Advanced testing and validation methods are needed to efficiently validate power equipment and controls in the contemporary complex environment and support the transition to a cleaner and sustainable energy system. Real-time hardware-in-the-loop (HIL) simulation has proven to be an effective method for validating and de-risking power system equipment in highly realistic, flexible, and repeatable conditions. Controller hardware-in-the-loop (CHIL) and power hardware-in-the-loop (PHIL) are the two main HIL simulation methods used in industry and academia that contribute to system-level testing enhancement by exploiting the flexibility of digital simulations in testing actual controllers and power equipment. This book addresses recent advances in real-time HIL simulation in several domains (also in new and promising areas), including technique improvements to promote its wider use. It is composed of 14 papers dealing with advances in HIL testing of power electronic converters, power system protection, modeling for real-time digital simulation, co-simulation, geographically distributed HIL, and multiphysics HIL, among other topics

Advanced Applications of Rapid Prototyping Technology in Modern Engineering

Rapid prototyping (RP) technology has been widely known and appreciated due to its flexible and customized manufacturing capabilities. The widely studied RP techniques include stereolithography apparatus (SLA), selective laser sintering (SLS), three-dimensional printing (3DP), fused deposition modeling (FDM), 3D plotting, solid ground curing (SGC), multiphase jet solidification (MJS), laminated object manufacturing (LOM). Different techniques are associated with different materials and/or processing principles and thus are devoted to specific applications. RP technology has no longer been only for prototype building rather has been extended for real industrial manufacturing solutions. Today, the RP technology has contributed to almost all engineering areas that include mechanical, materials, industrial, aerospace, electrical and most recently biomedical engineering. This book aims to present the advanced development of RP technologies in various engineering areas as the solutions to the real world engineering problems

Performance Improvement of the Inertial Sensors of Advanced Virgo Seismic Isolators with Digital Techniques

Gravitational waves, predicted on the basis of the General Relativity, are ripples in the curvature of space-time that propagate as a wave. The passage of a gravitational wave induces tiny oscillations in the relative separation between two test masses, that can be measured. Nevertheless these oscillations are extremely small, so that only a very sensitive detector is able to measure them. The Advanced Virgo project is a major upgrade of the 3 km-long interferometric gravitational wave detector Virgo, with the goal of increasing its sensitivity by about one order of magnitude in the whole detection band. We expect to have a maximum strain amplitude sensitivity of 4 × 10^−24 1/√Hz at ∼ 300 Hz. In other words this means that it will be able to detect a relative displacement between mirrors of about 10^−20 m, by averaging for one second. This sensitivity should allow to detect several tens of events per year. Among the various ongoing updates, an important improvement is represented by the new electronics used to control the Superattenuators, complex mechanical structures that isolate optical elements from seismic noise by a factor 10^15 at 1 Hz. Using the information of several inertial sensors, a digital control system keeps the structures as stable as possible. A new board for the Superattenuator control has been designed, that incorporates analog-to-digital and digital-to-analog converters, a Field Programmable Gate Array (FPGA) and a Digital Signal Processor (DSP) into a single unit. This board is enough to handle every single part of the Superattenuator inertial control. It performs the computation of feedback forces, and is used to synthesize sine wave to drive the coils of the inertial sensors, as well as to read their output. Furthermore it interfaces with all the other structures of Virgo. In this thesis I have studied the horizontal accelerometers, feedback-controlled sensors used in the Superattenuator inertial control to measure the seismic noise in the frequency band from DC to 100 Hz. Using the computing power of the new electronics (the new DSP has 8 cores and can compute 8.4 GFLOPS per core for double precision floating point indeed), I have designed a new control system for the accelerometers, exploiting the properties of a critically damped harmonic oscillator. This system allows to improve by about one order of magnitude the sensitivity of these sensors, with respect to the system used in Virgo, by reducing the root mean square of the force needed for the control by a factor 2. In this way, the accelerometer sensitivity can reach about 10^−9 (m/s^2)/√Hz at 1 Hz. In the last part of the thesis I have studied the Linear Variable Differential Transformer (LVDT), a kind of displacement sensor widely used in Superattenuator control. I have designed a system to read the output of LVDT using a FPGA. It consists of a Direct Digital Synthesizer (DDS) that is used both to drive the primary coil of the LVDT with a sine wave at 50 kHz, and then to demodulate the signal induced on the secondary coils, whose amplitude is modulated by a signal proportional to displacement. An algorithm, based on a Phase-Locked Loop (PLL), allows the detection of the phase shift of the signal induced on the secondary coils, and tunes the system in order to maximize the signal-to-noise ratio of the measurement of displacement

Advanced Knowledge Application in Practice

The integration and interdependency of the world economy leads towards the creation of a global market that offers more opportunities, but is also more complex and competitive than ever before. Therefore widespread research activity is necessary if one is to remain successful on the market. This book is the result of research and development activities from a number of researchers worldwide, covering concrete fields of research

Random Finite Sets Based Very Short-Term Solar Power Forecasting Through Cloud Tracking

Tracking clouds with a sky camera within a very short horizon below thirty seconds can be a solution to mitigate the effects of sunlight disruptions. A Probability Hypothesis Density (PHD) filter and a Cardinalised Probability Hypothesis Density (CPHD) filter were used on a set of pre-processed sky images. Both filters have been compared with the state-of-the-art methods for performance. It was found that both filters are suitable to perform very-short term irradiance forecasting