Stability analysis of recurrent neural networks using dissipativity

The purpose of this work is to describe how dissipativity theory can be used for the stability analysis of discrete-time recurrent neural networks and to propose a training algorithm for producing stable networks. Using dissipativity theory, we have found conditions for the globally asymptotic stability of equilibrium points of Layered Digital Dynamic Networks (LDDNs), a very general class of recurrent neural networks. The LDDNs are transformed into a standard interconnected system structure, and a fundamental theorem describing the stability of interconnected dissipative systems is applied. The theorem leads to several new sufficient conditions for the stability of equilibrium points for LDDNs. These conditions are demonstrated on several test problems and compared to previously proposed stability conditions. From these novel stability criteria, we propose a new algorithm to train stable recurrent neural networks. The standard mean square error performance index is modified to include stability criteria. This requires computation of the derivative of the maximum eigenvalue of a matrix with respect to neural network weights. The new training algorithm is tested on two examples of neural network-based model reference control systems, including a magnetic levitation system

Small gain theorems for large scale systems and construction of ISS Lyapunov functions

We consider interconnections of n nonlinear subsystems in the input-to-state stability (ISS) framework. For each subsystem an ISS Lyapunov function is given that treats the other subsystems as independent inputs. A gain matrix is used to encode the mutual dependencies of the systems in the network. Under a small gain assumption on the monotone operator induced by the gain matrix, a locally Lipschitz continuous ISS Lyapunov function is obtained constructively for the entire network by appropriately scaling the individual Lyapunov functions for the subsystems. The results are obtained in a general formulation of ISS, the cases of summation, maximization and separation with respect to external gains are obtained as corollaries.Comment: provisionally accepted by SIAM Journal on Control and Optimizatio

A novel technique for load frequency control of multi-area power systems

In this paper, an adaptive type-2 fuzzy controller is proposed to control the load frequency of a two-area power system based on descending gradient training and error back-propagation. The dynamics of the system are completely uncertain. The multilayer perceptron (MLP) artificial neural network structure is used to extract Jacobian and estimate the system model, and then, the estimated model is applied to the controller, online. A proportional–derivative (PD) controller is added to the type-2 fuzzy controller, which increases the stability and robustness of the system against disturbances. The adaptation, being real-time and independency of the system parameters are new features of the proposed controller. Carrying out simulations on New England 39-bus power system, the performance of the proposed controller is compared with the conventional PI, PID and internal model control based on PID (IMC-PID) controllers. Simulation results indicate that our proposed controller method outperforms the conventional controllers in terms of transient response and stability

Collective stability of networks of winner-take-all circuits

The neocortex has a remarkably uniform neuronal organization, suggesting that common principles of processing are employed throughout its extent. In particular, the patterns of connectivity observed in the superficial layers of the visual cortex are consistent with the recurrent excitation and inhibitory feedback required for cooperative-competitive circuits such as the soft winner-take-all (WTA). WTA circuits offer interesting computational properties such as selective amplification, signal restoration, and decision making. But, these properties depend on the signal gain derived from positive feedback, and so there is a critical trade-off between providing feedback strong enough to support the sophisticated computations, while maintaining overall circuit stability. We consider the question of how to reason about stability in very large distributed networks of such circuits. We approach this problem by approximating the regular cortical architecture as many interconnected cooperative-competitive modules. We demonstrate that by properly understanding the behavior of this small computational module, one can reason over the stability and convergence of very large networks composed of these modules. We obtain parameter ranges in which the WTA circuit operates in a high-gain regime, is stable, and can be aggregated arbitrarily to form large stable networks. We use nonlinear Contraction Theory to establish conditions for stability in the fully nonlinear case, and verify these solutions using numerical simulations. The derived bounds allow modes of operation in which the WTA network is multi-stable and exhibits state-dependent persistent activities. Our approach is sufficiently general to reason systematically about the stability of any network, biological or technological, composed of networks of small modules that express competition through shared inhibition.Comment: 7 Figure

Sliding-mode neuro-controller for uncertain systems

In this paper, a method that allows for the merger of the good features of sliding-mode control and neural network (NN) design is presented. Design is performed by applying an NN to minimize the cost function that is selected to depend on the distance from the sliding-mode manifold, thus providing that the NN controller enforces sliding-mode motion in a closed-loop system. It has been proven that the selected cost function has no local minima in controller parameter space, so under certain conditions, selection of the NN weights guarantees that the global minimum is reached, and then the sliding-mode conditions are satisfied; thus, closed-loop motion is robust against parameter changes and disturbances. For controller design, the system states and the nominal value of the control input matrix are used. The design for both multiple-input-multiple-output and single-input-single-output systems is discussed. Due to the structure of the (M)ADALINE network used in control calculation, the proposed algorithm can also be interpreted as a sliding-mode-based control parameter adaptation scheme. The controller performance is verified by experimental results