Deep neural networks for electromagnetics applied to optimization and inverse problems

This thesis explores the possibilities to complement full-wave electromagnetic solvers with fully-connected neural networks. We emphasize problems that must be solved a very large number of times in a limited parameter domain.We present and evaluate the normalization method ForwardNorm. ForwardNorm normalizes the outputs of the hidden layers of a neural network and enables training of very deep fully-connected neural networks. To minimize the number of samples needed train the very deep neural networks, we formulate a loss function that includes the misfit in (i) the output of the neural network and (ii) the derivatives of the output of the neural network with respect to its inputs. For certain combinations of input and output, we use continuum sensitivity analysis to compute these derivatives at a low computation cost. We also develop an auto-calibration method that simultaneously determines (i) a set of unknown amplification factors and (ii) the mean permittivity of an unknown medium under test. The method assumes that we have access to a set of measurements that are made a-priori for the purpose of characterization. The method is intended for on-line applications.We test the methods on four different test-problems. For the first three test-problems, we consider a type of microwave measurement-device intended for an inhomogeneous dielectric medium transported through a metal pipe. In the first test-problem, we train a neural network to determine the point-wise mean and variance of the permittivity of the inhomogeneous dielectric. The trained neural network is very computationally cheap to evaluate, which makes the method appealing for real-time applications. For the second test-problem, we apply the auto-calibration method to simultaneously determine (i) the mean permittivity in the pipe and (ii) a set of unknown amplification factors. For the third test-problem, we use a deep neural network to model the microwave measurement-device with a stochastic dielectric medium and estimate high-dimensional histograms. For the fourth test-problem, we train a deep neural network to model the frequency response of an H-plane waveguide filter as a function of its geometrical parameters. We then use the neural network to optimize the geometry of the filter to achieve pass-band characteristics under geometrical uncertainty

Dynamical Systems in Spiking Neuromorphic Hardware

Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case

Nonlinear Systems

Open Mathematics is a challenging notion for theoretical modeling, technical analysis, and numerical simulation in physics and mathematics, as well as in many other fields, as highly correlated nonlinear phenomena, evolving over a large range of time scales and length scales, control the underlying systems and processes in their spatiotemporal evolution. Indeed, available data, be they physical, biological, or financial, and technologically complex systems and stochastic systems, such as mechanical or electronic devices, can be managed from the same conceptual approach, both analytically and through computer simulation, using effective nonlinear dynamics methods. The aim of this Special Issue is to highlight papers that show the dynamics, control, optimization and applications of nonlinear systems. This has recently become an increasingly popular subject, with impressive growth concerning applications in engineering, economics, biology, and medicine, and can be considered a veritable contribution to the literature. Original papers relating to the objective presented above are especially welcome subjects. Potential topics include, but are not limited to: Stability analysis of discrete and continuous dynamical systems; Nonlinear dynamics in biological complex systems; Stability and stabilization of stochastic systems; Mathematical models in statistics and probability; Synchronization of oscillators and chaotic systems; Optimization methods of complex systems; Reliability modeling and system optimization; Computation and control over networked systems

Time series prediction by perturbed fuzzy model

This paper presents a fuzzy system approach to the prediction of nonlinear time series and dynamical systems based on a fuzzy model that includes its derivative information. The underlying mechanism governing the time series, expressed as a set of IF–THEN rules, is discovered by a modified structure of fuzzy system in order to capture the temporal series and its temporal derivative information. The task of predicting the future is carried out by a fuzzy predictor on the basis of the extracted rules and by the Taylor ODE solver method. We have applied the approach to the benchmark Mackey-Glass chaotic time series.This work was supported by the Portuguese Fundação para a Ciência e a Tecnologia (FCT) under grant POSI/SRI/41975/2001

Hole Cleaning And Cuttings Transportation Modelling And Optimization

Efficient hole cleaning in drilling operation is essential to ensure optimum rate of penetration. This complex problem involves simultaneous analysis of multiple parameters, including cuttings characteristics, fluid rheology and the geometry of the annulus space. For instance, accurate calculations of the equivalent circulation density (ECD) requires the effect of the mud density increase due to the cuttings’ concentration to be considered, which itself is a function of the settling velocity and the rate of penetration (ROP). Analytical models, lab experiments and numerical simulations have been used to determine the optimum flow rate for efficient hole cleaning. Most of these models are based on the drag coefficient-Reynolds number relationship, where both parameters are velocity dependent, making the calculation workflow to be implicit, tedious and time consuming. While several attempts have been made to present explicit correlations, precise equations covering a wide range of Reynolds numbers are not available.Terminal settling velocity was used in this research to determine the minimum required transportation velocity of drilling cuttings in the annulus space to ensure an optimal cleaning. The ROP also affects the hole cleaning as it defines the volume of the cuttings produced. We first used analytical models to investigate the effect of the cuttings size, density, and fluid properties as a function of wellbore deviation and circulation rate on hole cleaning efficiency. The results were compared with lab experiments using a slurry loop. The analytical models predict the critical velocities for lifting and rolling the cuttings particles based on the equilibrium cuttings bed height model and forces acting on a cuttings bed. For vertical sections of the wellbore, the critical transportation velocity showed to be proportional to the terminal settling velocity of the drill cuttings. Hence, we developed two new methods to predict the hindered terminal settling velocity due to the presence of wellbore and pipe walls and particle shape. We then used the Artificial Neural Network (ANN) algorithm and generated two models to predict the terminal velocity of drill cuttings and proppants considering the particles shape and the wall effect. The results of both analytical models and ANN were applied to estimate ECD. In addition, the drilling Mechanical Specific Energy (MSE) was calculated to determine the effect of different drilling parameters on hole cleaning and ECD. A new model was proposed for predicting the ECD in vertical and deviated wellbores that considers fluid and formation properties as well as wellbore and drill string geometry and drilling operational parameters. The developed model was used to study the effect of different drilling parameters on ECD and help engineers to optimize their operational parameters. The final step of this study was to investigate the effect of stabilizers geometry on hole cleaning. A total of more than 30 different designs of straight, straight with offset and helical blades geometries were built numerically and the results were compared. The reliability of the numerical simulation was confirmed against experimental and field data from the literature. The effect of size and shape of the stabilizer blades on the motion of the particles was investigated. Numerical simulation results showed that the straight blade geometry causes less disturbance to the cuttings transportation as compared to the straight with offset and helical blades, respectively

On the Application of PSpice for Localised Cloud Security

The work reported in this thesis commenced with a review of methods for creating random binary sequences for encoding data locally by the client before storing in the Cloud. The first method reviewed investigated evolutionary computing software which generated noise-producing functions from natural noise, a highly-speculative novel idea since noise is stochastic. Nevertheless, a function was created which generated noise to seed chaos oscillators which produced random binary sequences and this research led to a circuit-based one-time pad key chaos encoder for encrypting data. Circuit-based delay chaos oscillators, initialised with sampled electronic noise, were simulated in a linear circuit simulator called PSpice. Many simulation problems were encountered because of the nonlinear nature of chaos but were solved by creating new simulation parts, tools and simulation paradigms. Simulation data from a range of chaos sources was exported and analysed using Lyapunov analysis and identified two sources which produced one-time pad sequences with maximum entropy. This led to an encoding system which generated unlimited, infinitely-long period, unique random one-time pad encryption keys for plaintext data length matching. The keys were studied for maximum entropy and passed a suite of stringent internationally-accepted statistical tests for randomness. A prototype containing two delay chaos sources initialised by electronic noise was produced on a double-sided printed circuit board and produced more than 200 Mbits of OTPs. According to Vladimir Kotelnikov in 1941 and Claude Shannon in 1945, one-time pad sequences are theoretically-perfect and unbreakable, provided specific rules are adhered to. Two other techniques for generating random binary sequences were researched; a new circuit element, memristance was incorporated in a Chua chaos oscillator, and a fractional-order Lorenz chaos system with order less than three. Quantum computing will present many problems to cryptographic system security when existing systems are upgraded in the near future. The only existing encoding system that will resist cryptanalysis by this system is the unconditionally-secure one-time pad encryption

Fast recursive filters for simulating nonlinear dynamic systems

A fast and accurate computational scheme for simulating nonlinear dynamic systems is presented. The scheme assumes that the system can be represented by a combination of components of only two different types: first-order low-pass filters and static nonlinearities. The parameters of these filters and nonlinearities may depend on system variables, and the topology of the system may be complex, including feedback. Several examples taken from neuroscience are given: phototransduction, photopigment bleaching, and spike generation according to the Hodgkin-Huxley equations. The scheme uses two slightly different forms of autoregressive filters, with an implicit delay of zero for feedforward control and an implicit delay of half a sample distance for feedback control. On a fairly complex model of the macaque retinal horizontal cell it computes, for a given level of accuracy, 1-2 orders of magnitude faster than 4th-order Runge-Kutta. The computational scheme has minimal memory requirements, and is also suited for computation on a stream processor, such as a GPU (Graphical Processing Unit).Comment: 20 pages, 8 figures, 1 table. A comparison with 4th-order Runge-Kutta integration shows that the new algorithm is 1-2 orders of magnitude faster. The paper is in press now at Neural Computatio