Recurrent neural networks with fixed time convergence for linear and quadratic programming

In this paper, a new class of recurrent neural networks which solve linear and quadratic programs are presented. Their design is considered as a sliding mode control problem, where the network structure is based on the Karush-Kuhn-Tucker (KKT) optimality conditions with the KKT multipliers considered as control inputs to be implemented with fixed time stabilizing terms, instead of common used activation functions. Thus, the main feature of the proposed network is its fixed convergence time to the solution. That means, there is time independent to the initial conditions in which the network converges to the optimization solution. Simulations show the feasibility of the current approach

Recurrent neural networks with fixed time convergence for linear and quadratic programming

In this paper, a new class of recurrent neural networks which solve linear and quadratic programs are presented. Their design is considered as a sliding mode control problem, where the network structure is based on the Karush-Kuhn-Tucker (KKT) optimality conditions with the KKT multipliers considered as control inputs to be implemented with fixed time stabilizing terms, instead of common used activation functions. Thus, the main feature of the proposed network is its fixed convergence time to the solution. That means, there is time independent to the initial conditions in which the network converges to the optimization solution. Simulations show the feasibility of the current approach.Consejo Nacional de Ciencia y Tecnologí

A discontinuous recurrent neural network with predefined time convergence for solution of linear programming

The aim of this paper is to introduce a new recurrent neural network to solve linear programming. The main characteristic of the proposed scheme is its design based on the predefined-time stability. The predefined-time stability is a stronger form of finite-time stability which allows the a priori definition of a convergence time that does not depend on the network initial state. The network structure is based on the Karush-Kuhn-Tucker (KKT) conditions and the KKT multipliers are proposed as sliding mode control inputs. This selection yields to an one-layer recurrent neural network in which the only parameter to be tuned is the desired convergence time. With this features, the network can be easily scaled from a small to a higher dimension problem. The simulation of a simple example shows the feasibility of the current approach.Consejo Nacional de Ciencia y Tecnologí

ACTIVATION FUNCTIONS IN SINGLE HIDDEN LAYER FEED-FORWARD NEURAL NETWORKS

AbstractEspecially in the last decade, Artificial Intelligence (AI) has gained increasing popularity as the neural networks represent incredibly exciting and powerful machine learning-based techniques that can solve many real-time problems. The learning capability of such systems is directly related with the evaluation methods used. In this study, the effectiveness of the calculation parameters in a Single-Hidden Layer Feedforward Neural Networks (SLFNs) will be examined. We will present how important the selection of an activation function is in the learning stage. A lot of work is developed and presented for SLFNs up to now. Our study uses one of the most commonly known learning algorithms, which is Extreme Learning Machine (ELM). Main task of an activation function is to map the input value of a neural network to the output node with a high learning or achievement rate. However, determining the correct activation function is not as simple as thought. First we try to show the effect of the activation functions on different datasets and then we propose a method for selection process of it due to the characteristic of any dataset. The results show that this process is providing a remarkably better performance and learning rate in a sample neural network.Keywords: Machine Learning, SLFN, ELM

A neurodynamic optimization approach to constrained pseudoconvex optimization.

Guo, Zhishan.Thesis (M.Phil.)--Chinese University of Hong Kong, 2011.Includes bibliographical references (p. 71-82).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement i --- p.iiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Constrained Pseudoconvex Optimization --- p.1Chapter 1.2 --- Recurrent Neural Networks --- p.4Chapter 1.3 --- Thesis Organization --- p.7Chapter 2 --- Literature Review --- p.8Chapter 2.1 --- Pseudo convex Optimization --- p.8Chapter 2.2 --- Recurrent Neural Networks --- p.10Chapter 3 --- Model Description and Convergence Analysis --- p.17Chapter 3.1 --- Model Descriptions --- p.18Chapter 3.2 --- Global Convergence --- p.20Chapter 4 --- Numerical Examples --- p.27Chapter 4.1 --- Gaussian Optimization --- p.28Chapter 4.2 --- Quadratic Fractional Programming --- p.36Chapter 4.3 --- Nonlinear Convex Programming --- p.39Chapter 5 --- Real-time Data Reconciliation --- p.42Chapter 5.1 --- Introduction --- p.42Chapter 5.2 --- Theoretical Analysis and Performance Measurement --- p.44Chapter 5.3 --- Examples --- p.45Chapter 6 --- Real-time Portfolio Optimization --- p.53Chapter 6.1 --- Introduction --- p.53Chapter 6.2 --- Model Description --- p.54Chapter 6.3 --- Theoretical Analysis --- p.56Chapter 6.4 --- Illustrative Examples --- p.58Chapter 7 --- Conclusions and Future Works --- p.67Chapter 7.1 --- Concluding Remarks --- p.67Chapter 7.2 --- Future Works --- p.68Chapter A --- Publication List --- p.69Bibliography --- p.7

Distributed model predictive control of linear systems with coupled constraints based on collective neurodynamic optimization

© Springer Nature Switzerland AG 2018. Distributed model predictive control explores an array of local predictive controllers that synthesize the control of subsystems independently yet they communicate to efficiently cooperate in achieving the closed-loop control performance. Distributed model predictive control problems naturally result in sequential distributed optimization problems that require real-time solution. This paper presents a collective neurodynamic approach to design and implement the distributed model predictive control of linear systems in the presence of globally coupled constraints. For each subsystem, a neurodynamic model minimizes its cost function using local information only. According to the communication topology of the network, neurodynamic models share information to their neighbours to reach consensus on the optimal control actions to be carried out. The collective neurodynamic models are proven to guarantee the global optimality of the model predictive control system

Neurodynamic approaches to model predictive control.

Pan, Yunpeng.Thesis (M.Phil.)--Chinese University of Hong Kong, 2009.Includes bibliographical references (p. 98-107).Abstract also in Chinese.Abstract --- p.ip.iiiAcknowledgement --- p.ivChapter 1 --- Introduction --- p.2Chapter 1.1 --- Model Predictive Control --- p.2Chapter 1.2 --- Neural Networks --- p.3Chapter 1.3 --- Existing studies --- p.6Chapter 1.4 --- Thesis structure --- p.7Chapter 2 --- Two Recurrent Neural Networks Approaches to Linear Model Predictive Control --- p.9Chapter 2.1 --- Problem Formulation --- p.9Chapter 2.1.1 --- Quadratic Programming Formulation --- p.10Chapter 2.1.2 --- Linear Programming Formulation --- p.13Chapter 2.2 --- Neural Network Approaches --- p.15Chapter 2.2.1 --- Neural Network Model 1 --- p.15Chapter 2.2.2 --- Neural Network Model 2 --- p.16Chapter 2.2.3 --- Control Scheme --- p.17Chapter 2.3 --- Simulation Results --- p.18Chapter 3 --- Model Predictive Control for Nonlinear Affine Systems Based on the Simplified Dual Neural Network --- p.22Chapter 3.1 --- Problem Formulation --- p.22Chapter 3.2 --- A Neural Network Approach --- p.25Chapter 3.2.1 --- The Simplified Dual Network --- p.26Chapter 3.2.2 --- RNN-based MPC Scheme --- p.28Chapter 3.3 --- Simulation Results --- p.28Chapter 3.3.1 --- Example 1 --- p.28Chapter 3.3.2 --- Example 2 --- p.29Chapter 3.3.3 --- Example 3 --- p.33Chapter 4 --- Nonlinear Model Predictive Control Using a Recurrent Neural Network --- p.36Chapter 4.1 --- Problem Formulation --- p.36Chapter 4.2 --- A Recurrent Neural Network Approach --- p.40Chapter 4.2.1 --- Neural Network Model --- p.40Chapter 4.2.2 --- Learning Algorithm --- p.41Chapter 4.2.3 --- Control Scheme --- p.41Chapter 4.3 --- Application to Mobile Robot Tracking --- p.42Chapter 4.3.1 --- Example 1 --- p.44Chapter 4.3/2 --- Example 2 --- p.44Chapter 4.3.3 --- Example 3 --- p.46Chapter 4.3.4 --- Example 4 --- p.48Chapter 5 --- Model Predictive Control of Unknown Nonlinear Dynamic Sys- tems Based on Recurrent Neural Networks --- p.50Chapter 5.1 --- MPC System Description --- p.51Chapter 5.1.1 --- Model Predictive Control --- p.51Chapter 5.1.2 --- Dynamical System Identification --- p.52Chapter 5.2 --- Problem Formulation --- p.54Chapter 5.3 --- Dynamic Optimization --- p.58Chapter 5.3.1 --- The Simplified Dual Neural Network --- p.59Chapter 5.3.2 --- A Recursive Learning Algorithm --- p.60Chapter 5.3.3 --- Convergence Analysis --- p.61Chapter 5.4 --- RNN-based MPC Scheme --- p.65Chapter 5.5 --- Simulation Results --- p.67Chapter 5.5.1 --- Example 1 --- p.67Chapter 5.5.2 --- Example 2 --- p.68Chapter 5.5.3 --- Example 3 --- p.76Chapter 6 --- Model Predictive Control for Systems With Bounded Uncertainties Using a Discrete-Time Recurrent Neural Network --- p.81Chapter 6.1 --- Problem Formulation --- p.82Chapter 6.1.1 --- Process Model --- p.82Chapter 6.1.2 --- Robust. MPC Design --- p.82Chapter 6.2 --- Recurrent Neural Network Approach --- p.86Chapter 6.2.1 --- Neural Network Model --- p.86Chapter 6.2.2 --- Convergence Analysis --- p.88Chapter 6.2.3 --- Control Scheme --- p.90Chapter 6.3 --- Simulation Results --- p.91Chapter 7 --- Summary and future works --- p.95Chapter 7.1 --- Summary --- p.95Chapter 7.2 --- Future works --- p.96Bibliography --- p.9

Fault Diagnosis and Fault-Tolerant Control of Unmanned Aerial Vehicles

With the increasing demand for unmanned aerial vehicles (UAVs) in both military and civilian applications, critical safety issues need to be specially considered in order to make better and wider use of them. UAVs are usually employed to work in hazardous and complex environments, which may seriously threaten the safety and reliability of UAVs. Therefore, the safety and reliability of UAVs are becoming imperative for development of advanced intelligent control systems. The key challenge now is the lack of fully autonomous and reliable control techniques in face of different operation conditions and sophisticated environments. Further development of unmanned aerial vehicle (UAV) control systems is required to be reliable in the presence of system component faults and to be insensitive to model uncertainties and external environmental disturbances. This thesis research aims to design and develop novel control schemes for UAVs with consideration of all the factors that may threaten their safety and reliability. A novel adaptive sliding mode control (SMC) strategy is proposed to accommodate model uncertainties and actuator faults for an unmanned quadrotor helicopter. Compared with the existing adaptive SMC strategies in the literature, the proposed adaptive scheme can tolerate larger actuator faults without stimulating control chattering due to the use of adaptation parameters in both continuous and discontinuous control parts. Furthermore, a fuzzy logic-based boundary layer and a nonlinear disturbance observer are synthesized to further improve the capability of the designed control scheme for tolerating model uncertainties, actuator faults, and unknown external disturbances while preventing overestimation of the adaptive control parameters and suppressing the control chattering effect. Then, a cost-effective fault estimation scheme with a parallel bank of recurrent neural networks (RNNs) is proposed to accurately estimate actuator fault magnitude and an active fault-tolerant control (FTC) framework is established for a closed-loop quadrotor helicopter system. Finally, a reconfigurable control allocation approach is combined with adaptive SMC to achieve the capability of tolerating complete actuator failures with application to a modified octorotor helicopter. The significance of this proposed control scheme is that the stability of the closed-loop system is theoretically guaranteed in the presence of both single and simultaneous actuator faults