A Numerical Slow Manifold Approach to Model Reduction for Optimal Control of Multiple Time Scale ODE

Time scale separation is a natural property of many control systems that can be ex- ploited, theoretically and numerically. We present a numerical scheme to solve optimal control problems with considerable time scale separation that is based on a model reduction approach that does not need the system to be explicitly stated in singularly perturbed form. We present examples that highlight the advantages and disadvantages of the method

Extended Trust-Tech Methodology For Nonlinear Optimization: Analyses, Methods And Applications

Many theoretical and practical problems can be formulated as a global optimization problem. Traditional local optimization methods can only attain a local optimal solution and be entrapped in the local optimal solution; while existing global optimization algorithms usually sparsely approximates the global optimal solution in a stochastic manner. In contrast, the transformation under stability-retaining equilibrium characterization (TRUST-TECH) methodology prevails over existing algorithms due to its capability of locating multiple, if not all, local optimal solutions to the optimization problem deterministically and systematically in a tier-by-tier manner. The TRUST-TECH methodology was developed to solve unconstrained and constrained nonlinear optimization problems. This work extends the TRUST-TECH methodology by incorporating new analytical results, developing new solution methods and solving new problems in practical applications. This work first provides analytical results regarding the invariance of partial stability region in quasi-gradient systems. Our motivation is to resolve numerical difficulties arising in implementations of trajectory based methods, including TRUST-TECH. Improved algorithms were developed to resolve these issues by altering the original problem to speed-up movement of the trajectory. However, such operations can lead the trajectory converge to a different solution, which could be undesired under specific situations. This work attempts to answer the question regarding invariant convergence for a special class of numerical operations whose dynamical behaviours can be characterized by a quasi-gradient dynamical system. To this end, we study relationship between a gradient dynamical system and its associated quasi-gradient system and reveal the invariance of partial stability region in the quasi-gradient system. These analytical results lead to methods for checking invariant convergence of the trajectory starting from a given point in the quasi-gradient system and the algorithm to maintain invariant convergence. This work also develops new solution methods to enhance TRUST-TECH's capability of solving constrained nonlinear optimization problems and applies them to solve practical problems arising in different applications. Specifically, TRUST-TECH based methods are first developed for feasibility computation and restoration and are applied to power system applications, including power flow computation and feasibility restoration for infeasible optimal power flow problems. Indeed, a unified framework based on TRUST-TECH is introduced for analysing feasibility and infeasibility for nonlinear problems. Secondly, the TRUST-TECH based interior point method (TT-IPM) and the reduced projected gradient method are developed to better tackle constrained nonlinear optimization problems. As application, the TT-IPM method is used to solve mixed-integer nonlinear programs (MINLPs). Finally, this work develops the ensemble of optimal, input-pruned neural networks using TRUST-TECH (ELITE) method for constructing high-quality neural network ensembles and applies ELITE to build a short-term load forecaster named ELITE-STLF with promising performance. Possible extensions of the TRUST-TECH methodology to a much broader range of optimization models, including multi-objective optimization and variational optimization, are suggested for future research efforts

Neural Networks: Training and Application to Nonlinear System Identification and Control

This dissertation investigates training neural networks for system identification and classification. The research contains two main contributions as follow:1. Reducing number of hidden layer nodes using a feedforward componentThis research reduces the number of hidden layer nodes and training time of neural networks to make them more suited to online identification and control applications by adding a parallel feedforward component. Implementing the feedforward component with a wavelet neural network and an echo state network provides good models for nonlinear systems.The wavelet neural network with feedforward component along with model predictive controller can reliably identify and control a seismically isolated structure during earthquake. The network model provides the predictions for model predictive control. Simulations of a 5-story seismically isolated structure with conventional lead-rubber bearings showed significant reductions of all response amplitudes for both near-field (pulse) and far-field ground motions, including reduced deformations along with corresponding reduction in acceleration response. The controller effectively regulated the apparent stiffness at the isolation level. The approach is also applied to the online identification and control of an unmanned vehicle. Lyapunov theory is used to prove the stability of the wavelet neural network and the model predictive controller. 2. Training neural networks using trajectory based optimization approachesTraining neural networks is a nonlinear non-convex optimization problem to determine the weights of the neural network. Traditional training algorithms can be inefficient and can get trapped in local minima. Two global optimization approaches are adapted to train neural networks and avoid the local minima problem. Lyapunov theory is used to prove the stability of the proposed methodology and its convergence in the presence of measurement errors. The first approach transforms the constraint satisfaction problem into unconstrained optimization. The constraints define a quotient gradient system (QGS) whose stable equilibrium points are local minima of the unconstrained optimization. The QGS is integrated to determine local minima and the local minimum with the best generalization performance is chosen as the optimal solution. The second approach uses the QGS together with a projected gradient system (PGS). The PGS is a nonlinear dynamical system, defined based on the optimization problem that searches the components of the feasible region for solutions. Lyapunov theory is used to prove the stability of PGS and QGS and their stability under presence of measurement noise