A Novel Neuroglial Architecture for Modelling Singular Perturbation System

This work develops a new modular architecture that emulates a recently-discovered biological paradigm. It originates from the human brain where the information flows along two different pathways and is processed along two time scales: one is a fast neural network (NN) and the other is a slow network called the glial network (GN). It was found that the neural network is powered and controlled by the glial network. Based on our biological knowledge of glial cells and the powerful concept of modularity, a novel approach called artificial neuroglial Network (ANGN) was designed and an algorithm based on different concepts of modularity was also developed. The implementation is based on the notion of multi-time scale systems. Validation is performed through an asynchronous machine (ASM) modeled in the standard singularly perturbed form. We apply the geometrical approach, based on Gerschgorin’s circle theorem (GCT), to separate the fast and slow variables, as well as the singular perturbation method (SPM) to determine the reduced models. This new architecture makes it possible to obtain smaller networks with less complexity and better performance

Connections Between Adaptive Control and Optimization in Machine Learning

This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.Comment: 18 page

Identification and Control of Nonlinear Singularly Perturbed Systems Using Multi-time-scale Neural Networks

Many industrial systems are nonlinear with "slow" and "fast" dynamics because of the presence of some ``parasitic" parameters such as small time constants, resistances, inductances, capacitances, masses and moments of inertia. These systems are usually labeled as "singularly perturbed" or ``multi-time-scale" systems. Singular perturbation theory has been proved to be a useful tool to control and analyze singularly perturbed systems if the full knowledge of the system model parameters is available. However, the accurate and faithful mathematical models of those systems are usually difficult to obtain due to the uncertainties and nonlinearities. To obtain the accurate system models, in this research, a new identification scheme for the discrete time nonlinear singularly perturbed systems using multi-time-scale neural network and optimal bounded ellipsoid method is proposed firstly. Compared with other gradient descent based identification schemes, the new identification method proposed in this research can achieve faster convergence and higher accuracy due to the adaptively adjusted learning gain. Later, the optimal bounded ellipsoid based identification method for discrete time systems is extended to the identification of continuous singularly perturbed systems. Subsequently, by adding two additional terms in the weight's updating laws, a modified identification scheme is proposed to guarantee the effectiveness of the identification algorithm during the whole identification process. Lastly, through introducing some filtered variables, a robust neural network training algorithm is proposed for the system identification problem subjected to measurement noises. Based on the identification results, the singular perturbation theory is introduced to decompose a high order multi-time-scale system into two low order subsystems -- the reduced slow subsystem and the reduced fast subsystem. Then, two controllers are designed for the two subsystems separately. By using the singular perturbation theory, an adaptive controller for a regulation problem is designed in this research firstly. Because the system order is reduced, the adaptive controller proposed in this research has a simpler structure and requires much less computational resources, compared with other conventional controllers. Afterward, an indirect adaptive controller is proposed for solving the trajectory tracking problem. The stability of both identification and control schemes are analyzed through the Lyapunov approach, and the effectiveness of the identification and control algorithms are demonstrated using simulations and experiments

Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202

Nonlinear system identification and control using dynamic multi-time scales neural networks

In this thesis, on-line identification algorithm and adaptive control design are proposed for nonlinear singularly perturbed systems which are represented by dynamic neural network model with multi-time scales. A novel on-line identification law for the Neural Network weights and linear part matrices of the model has been developed to minimize the identification errors. Based on the identification results, an adaptive controller is developed to achieve trajectory tracking. The Lyapunov synthesis method is used to conduct stability analysis for both identification algorithm and control design. To further enhance the stability and performance of the control system, an improved . dynamic neural network model is proposed by replacing all the output signals from the plant with the state variables of the neural network. Accordingly, the updating laws are modified with a dead-zone function to prevent parameter drifting. By combining feedback linearization with one of three classical control methods such as direct compensator, sliding mode controller or energy function compensation scheme, three different adaptive controllers have been proposed for trajectory tracking. New Lyapunov function analysis method is applied for the stability analysis of the improved identification algorithm and three control systems. Extensive simulation results are provided to support the effectiveness of the proposed identification algorithms and control systems for both dynamic NN models

Neural networks in geophysical applications

Neural networks are increasingly popular in geophysics. Because they are universal approximators, these tools can approximate any continuous function with an arbitrary precision. Hence, they may yield important contributions to finding solutions to a variety of geophysical applications. However, knowledge of many methods and techniques recently developed to increase the performance and to facilitate the use of neural networks does not seem to be widespread in the geophysical community. Therefore, the power of these tools has not yet been explored to their full extent. In this paper, techniques are described for faster training, better overall performance, i.e., generalization,and the automatic estimation of network size and architecture

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique