Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey

Dynamic networks are used in a wide range of fields, including social network analysis, recommender systems, and epidemiology. Representing complex networks as structures changing over time allow network models to leverage not only structural but also temporal patterns. However, as dynamic network literature stems from diverse fields and makes use of inconsistent terminology, it is challenging to navigate. Meanwhile, graph neural networks (GNNs) have gained a lot of attention in recent years for their ability to perform well on a range of network science tasks, such as link prediction and node classification. Despite the popularity of graph neural networks and the proven benefits of dynamic network models, there has been little focus on graph neural networks for dynamic networks. To address the challenges resulting from the fact that this research crosses diverse fields as well as to survey dynamic graph neural networks, this work is split into two main parts. First, to address the ambiguity of the dynamic network terminology we establish a foundation of dynamic networks with consistent, detailed terminology and notation. Second, we present a comprehensive survey of dynamic graph neural network models using the proposed terminologyComment: 28 pages, 9 figures, 8 table

System Identification of multi-rotor UAVs using echo state networks

Controller design for aircraft with unusual configurations presents unique challenges, particularly in extracting valid mathematical models of the MRUAVs behaviour. System Identification is a collection of techniques for extracting an accurate mathematical model of a dynamic system from experimental input-output data. This can entail parameter identification only (known as grey-box modelling) or more generally full parameter/structural identification of the nonlinear mapping (known as black-box). In this paper we propose a new method for black-box identification of the non-linear dynamic model of a small MRUAV using Echo State Networks (ESN), a novel approach to train Recurrent Neural Networks (RNN)

Modelling of a post-combustion CO₂ capture process using neural networks

This paper presents a study of modelling post-combustion CO₂ capture process using bootstrap aggregated neural networks. The neural network models predict CO₂ capture rate and CO₂ capture level using the following variables as model inputs: inlet flue gas flow rate, CO₂ concentration in inlet flue gas, pressure of flue gas, temperature of flue gas, lean solvent flow rate, MEA concentration and temperature of lean solvent. In order to enhance model accuracy and reliability, multiple feedforward neural network models are developed from bootstrap re-sampling replications of the original training data and are combined. Bootstrap aggregated model can offer more accurate predictions than a single neural network, as well as provide model prediction confidence bounds. Simulated CO₂ capture process operation data from gPROMS simulation are used to build and verify neural network models. Both neural network static and dynamic models are developed and they offer accurate predictions on unseen validation data. The developed neural network models can then be used in the optimisation of the CO₂ capture process

Nonlinear dynamic modelling of flexible beam structures using neural networks

This paper investigates the utilisution of back propagation neural networlu (NNs) for modelling flexible beam structures infixed-free mode; a simple repsentation of an aircrufr wing or robot arm. A comparative performance of the NN model and conventional recursive least square scheme, in characterising the system is carried out in the time and frequency domains. Simulated results demonstrate that using NN approach the system is modelled better than with the conventional linear modelling approach. The developed neuro-modelling approach will firther be utilized in the design and implementation of suitable controllers, for vibration slippression in such system

BLACK BOX EFFICIENCY MODELLING OF AN ELECTRIC DRIVE UNIT UTILIZING METHODS OF MACHINE LEARNING

The increasing electrification of powertrains leads to increased demands for the test technology to ensure the required functions. For conventional test rigs in particular, it is necessary to have knowledge of the test technology's capabilities that can be applied in practical testing. Modelling enables early knowledge of the test rigs dynamic capabilities and the feasibility of planned testing scenarios. This paper describes the modelling of complex subsystems by experimental modelling with artificial neural networks taking transmission efficiency as an example. For data generation, the experimental design and execution is described. The generated data is pre-processed with suitable methods and optimized for the neural networks. Modelling is executed with different variants of the inputs as well as different algorithms. The variants compare and compete with each other. The most suitable variant is validated using statistical methods and other adequate techniques. The result represents reality well and enables the performance investigation of the test systems in a realistic manner

Demand forecasting for fast-moving products in grocery retail

Demand forecasting is a critically important task in grocery retail. Accurate forecasts allow the retail companies to reduce their product spoilage, as well as maximize their profits. Fast-moving products, or products with a lot of sales and fast turnover, are particularly important to forecast accurately due to their high sales volumes. We investigate dynamic harmonic regression, Poisson GLM with elastic net, MLP and two-layer LSTM in fast-moving product demand forecasting against the naive seasonal forecasting baseline. We evaluate two modes of seasonality modelling in neural networks: Fourier series against seasonal decomposition. We specify the full procedure for comparing forecasting models in a collection of product-location sales time series, involving two-stage cross-validation, and careful hyperparameter selection. We use Halton sequences for neural network hyperparameter selection. We evaluate the model results in demand forecasting using hypothesis testing, bootstrapping, and rank comparison methods. The experimental results suggest that the dynamic harmonic regression produces superior results in comparison to Poisson GLM, MLP and two-layer LSTM models for demand forecasting in fast-moving products with long sales histories. We additionally show that deseasonalization results in better forecasts in comparison to Fourier seasonality modelling in neural networks

Global models of dynamic complex systems – modelling using the multilayer neural networks

In this paper, global models of dynamic complex systems using the neural networks isdiscussed. The description of a complex system is given by a description of each system elementand structure. As a model the multilayer neural networks with the tapped delay line (TDL), whichhave the same structure as a complex system, are accepted. Two approaches, a global model and aglobal model with the quality local model taken into account are proposed.To learn global models the modified back-propagation algorithms have been developed for theunique structure of the complex model. To model dynamic simple plants, of which the complexsystem is composed, a series-parallel model of identification using the feedforward network withthe tapped delay line (TDL) and the feedback loops, in which the gradient can be calculated bymeans of the simpler static back-propagation method is proposed. Computer simulations wereperformed for the dynamic complex system, which consists of two dynamic nonlinear simpleplants connected in series, described by means of nonlinear difference equations

Modelling of river discharges using neural networks derived from support vector regression

Neural networks are often used to model complex and nonlinear systems, as they can approximate nonlinear systems with arbitrary accuracy and can be trained from data. Amongst the neural networks, Associative Memory Networks (AMNs) are often used, since they are less computation intensive, and yet good generalization results can be obtained. However, this can only be achieved if the structure of the AMNs is suitably chosen. An approach to choose the structure of the AMNs is to use the Support Vectors (SVs) obtained from the Support Vector Machines. The SVs are obtained from a constrained optimization for a given data set and an error bound. For convenience, this class of AMNs is referred to as the Support Vector Neural Networks (SVNNs). In this paper, the modelling of river discharges with rainfall as input using the SVNN is presented, from which the nonlinear dynamic relationship between rainfall and river discharges is obtained. The prediction of river discharges from the SVNN can give early warning of severe river discharges when there are heavy rainfalls.published_or_final_versio