Neural Networks: Training and Application to Nonlinear System Identification and Control

This dissertation investigates training neural networks for system identification and classification. The research contains two main contributions as follow:1. Reducing number of hidden layer nodes using a feedforward componentThis research reduces the number of hidden layer nodes and training time of neural networks to make them more suited to online identification and control applications by adding a parallel feedforward component. Implementing the feedforward component with a wavelet neural network and an echo state network provides good models for nonlinear systems.The wavelet neural network with feedforward component along with model predictive controller can reliably identify and control a seismically isolated structure during earthquake. The network model provides the predictions for model predictive control. Simulations of a 5-story seismically isolated structure with conventional lead-rubber bearings showed significant reductions of all response amplitudes for both near-field (pulse) and far-field ground motions, including reduced deformations along with corresponding reduction in acceleration response. The controller effectively regulated the apparent stiffness at the isolation level. The approach is also applied to the online identification and control of an unmanned vehicle. Lyapunov theory is used to prove the stability of the wavelet neural network and the model predictive controller. 2. Training neural networks using trajectory based optimization approachesTraining neural networks is a nonlinear non-convex optimization problem to determine the weights of the neural network. Traditional training algorithms can be inefficient and can get trapped in local minima. Two global optimization approaches are adapted to train neural networks and avoid the local minima problem. Lyapunov theory is used to prove the stability of the proposed methodology and its convergence in the presence of measurement errors. The first approach transforms the constraint satisfaction problem into unconstrained optimization. The constraints define a quotient gradient system (QGS) whose stable equilibrium points are local minima of the unconstrained optimization. The QGS is integrated to determine local minima and the local minimum with the best generalization performance is chosen as the optimal solution. The second approach uses the QGS together with a projected gradient system (PGS). The PGS is a nonlinear dynamical system, defined based on the optimization problem that searches the components of the feasible region for solutions. Lyapunov theory is used to prove the stability of PGS and QGS and their stability under presence of measurement noise

Advances in Computer Science and Engineering

The book Advances in Computer Science and Engineering constitutes the revised selection of 23 chapters written by scientists and researchers from all over the world. The chapters cover topics in the scientific fields of Applied Computing Techniques, Innovations in Mechanical Engineering, Electrical Engineering and Applications and Advances in Applied Modeling

Controlling Chaotic Maps using Next-Generation Reservoir Computing

In this work, we combine nonlinear system control techniques with next-generation reservoir computing, a best-in-class machine learning approach for predicting the behavior of dynamical systems. We demonstrate the performance of the controller in a series of control tasks for the chaotic H\'enon map, including controlling the system between unstable fixed-points, stabilizing the system to higher order periodic orbits, and to an arbitrary desired state. We show that our controller succeeds in these tasks, requires only 10 data points for training, can control the system to a desired trajectory in a single iteration, and is robust to noise and modeling error.Comment: 9 pages, 8 figure

Embedding and approximation theorems for echo state networks

Echo State Networks (ESNs) are a class of single layer recurrent neural networks that have enjoyed recent attention. In this paper we prove that a suitable ESN, trained on a series of measurements of an invertible dynamical system, induces a C1 map from the dynamical system's phase space to the ESN's reservoir space. We call this the Echo State Map. We then prove that the Echo State Map is generically an embedding with positive probability. Under additional mild assumptions, we further conjecture that the Echo State Map is almost surely an embedding. For sufficiently large, and specially structured, but still randomly generated ESNs, we prove that there exists a linear readout layer that allows the ESN to predict the next observation of a dynamical system arbitrarily well. Consequently, if the dynamical system under observation is structurally stable then the trained ESN will exhibit dynamics that are topologically conjugate to the future behaviour of the observed dynamical system. Our theoretical results connect the theory of ESNs to the delay-embedding literature for dynamical systems, and are supported by numerical evidence from simulations of the traditional Lorenz equations. The simulations confirm that, from a one dimensional observation function, an ESN can accurately infer a range of geometric and topological features of the dynamics such as the eigenvalues of equilibrium points, Lyapunov exponents and homology groups.Comment: 24 pages, 9 figure

Dynamical Systems

Complex systems are pervasive in many areas of science integrated in our daily lives. Examples include financial markets, highway transportation networks, telecommunication networks, world and country economies, social networks, immunological systems, living organisms, computational systems and electrical and mechanical structures. Complex systems are often composed of a large number of interconnected and interacting entities, exhibiting much richer global scale dynamics than the properties and behavior of individual entities. Complex systems are studied in many areas of natural sciences, social sciences, engineering and mathematical sciences. This special issue therefore intends to contribute towards the dissemination of the multifaceted concepts in accepted use by the scientific community. We hope readers enjoy this pertinent selection of papers which represents relevant examples of the state of the art in present day research. [...

Artificial Ontogenies: A Computational Model of the Control and Evolution of Development

Understanding the behaviour of biological systems is a challenging task. Gene regulation, development and evolution are each a product of nonlinear interactions between many individual agents: genes, cells or organisms. Moreover, these three processes are not isolated, but interact with one another in an important fashion. The development of an organism involves complex patterns of dynamic behaviour at the genetic level. The gene networks that produce this behaviour are subject to mutations that can alter the course of development, resulting in the production of novel morphologies. Evolution occurs when these novel morphologies are favoured by natural selection and survive to pass on their genes to future generations. Computational models can assist us to understand biological systems by providing a framework within which their behaviour can be explored. Many natural processes, including gene regulation and development, have a computational element to their control. Constructing formal models of these systems enables their behaviour to be simulated, observed and quantified on a scale not otherwise feasible. This thesis uses a computational simulation methodology to explore the relationship between development and evolution. An important question in evolutionary biology is how to explain the direction of evolution. Conventional explanations of evolutionary history have focused on the role of natural selection in orienting evolution. More recently, it has been argued that the nature of development, and the way it changes in response to mutation, may also be a significant factor. A network-lineage model of artificial ontogenies is described that incorporates a developmental mapping between the dynamics of a gene network and a cell lineage representation of a phenotype. Three series of simulation studies are reported, exploring: (a) the relationship between the structure of a gene network and its dynamic behaviour; (b) the characteristic distributions of ontogenies and phenotypes generated by the dynamics of gene networks; (c) the effect of these characteristic distributions on the evolution of ontogeny. The results of these studies indicate that the model networks are capable of generating a diverse range of stable behaviours, and possess a small yet significant sensitivity to perturbation. In the context of developmental control, the intrinsic dynamics of the model networks predispose the production of ontogenies with a modular, quasi-systematic structure. This predisposition is reflected in the structure of variation available for selection in an adaptive search process, resulting in the evolution of ontogenies biased towards simplicity. These results suggest a possible explanation for the levels of ontogenetic complexity observed in biological organisms: that they may be a product of the network architecture of developmental control. By quantifying complexity, variation and bias, the network-lineage model described in this thesis provides a computational method for investigating the effects of development on the direction of evolution. In doing so, it establishes a viable framework for simulating computational aspects of complex biological systems

Preventing premature convergence and proving the optimality in evolutionary algorithms

http://ea2013.inria.fr//proceedings.pdfInternational audienceEvolutionary Algorithms (EA) usually carry out an efficient exploration of the search-space, but get often trapped in local minima and do not prove the optimality of the solution. Interval-based techniques, on the other hand, yield a numerical proof of optimality of the solution. However, they may fail to converge within a reasonable time due to their inability to quickly compute a good approximation of the global minimum and their exponential complexity. The contribution of this paper is a hybrid algorithm called Charibde in which a particular EA, Differential Evolution, cooperates with a Branch and Bound algorithm endowed with interval propagation techniques. It prevents premature convergence toward local optima and outperforms both deterministic and stochastic existing approaches. We demonstrate its efficiency on a benchmark of highly multimodal problems, for which we provide previously unknown global minima and certification of optimality