Value Function Estimation in Optimal Control via Takagi-Sugeno Models and Linear Programming

[ES] La presente Tesis emplea técnicas de programación dinámica y aprendizaje por refuerzo para el control de sistemas no lineales en espacios discretos y continuos. Inicialmente se realiza una revisión de los conceptos básicos de programación dinámica y aprendizaje por refuerzo para sistemas con un número finito de estados. Se analiza la extensión de estas técnicas mediante el uso de funciones de aproximación que permiten ampliar su aplicabilidad a sistemas con un gran número de estados o sistemas continuos. Las contribuciones de la Tesis son: -Se presenta una metodología que combina identificación y ajuste de la función Q, que incluye la identificación de un modelo Takagi-Sugeno, el cálculo de controladores subóptimos a partir de desigualdades matriciales lineales y el consiguiente ajuste basado en datos de la función Q a través de una optimización monotónica. -Se propone una metodología para el aprendizaje de controladores utilizando programación dinámica aproximada a través de programación lineal. La metodología hace que ADP-LP funcione en aplicaciones prácticas de control con estados y acciones continuos. La metodología propuesta estima una cota inferior y superior de la función de valor óptima a través de aproximadores funcionales. Se establecen pautas para los datos y la regularización de regresores con el fin de obtener resultados satisfactorios evitando soluciones no acotadas o mal condicionadas. -Se plantea una metodología bajo el enfoque de programación lineal aplicada a programación dinámica aproximada para obtener una mejor aproximación de la función de valor óptima en una determinada región del espacio de estados. La metodología propone aprender gradualmente una política utilizando datos disponibles sólo en la región de exploración. La exploración incrementa progresivamente la región de aprendizaje hasta obtener una política convergida.[CA] La present Tesi empra tècniques de programació dinàmica i aprenentatge per reforç per al control de sistemes no lineals en espais discrets i continus. Inicialment es realitza una revisió dels conceptes bàsics de programació dinàmica i aprenentatge per reforç per a sistemes amb un nombre finit d'estats. S'analitza l'extensió d'aquestes tècniques mitjançant l'ús de funcions d'aproximació que permeten ampliar la seua aplicabilitat a sistemes amb un gran nombre d'estats o sistemes continus. Les contribucions de la Tesi són: -Es presenta una metodologia que combina identificació i ajust de la funció Q, que inclou la identificació d'un model Takagi-Sugeno, el càlcul de controladors subòptims a partir de desigualtats matricials lineals i el consegüent ajust basat en dades de la funció Q a través d'una optimització monotónica. -Es proposa una metodologia per a l'aprenentatge de controladors utilitzant programació dinàmica aproximada a través de programació lineal. La metodologia fa que ADP-LP funcione en aplicacions pràctiques de control amb estats i accions continus. La metodologia proposada estima una cota inferior i superior de la funció de valor òptima a través de aproximadores funcionals. S'estableixen pautes per a les dades i la regularització de regresores amb la finalitat d'obtenir resultats satisfactoris evitant solucions no fitades o mal condicionades. -Es planteja una metodologia sota l'enfocament de programació lineal aplicada a programació dinàmica aproximada per a obtenir una millor aproximació de la funció de valor òptima en una determinada regió de l'espai d'estats. La metodologia proposa aprendre gradualment una política utilitzant dades disponibles només a la regió d'exploració. L'exploració incrementa progressivament la regió d'aprenentatge fins a obtenir una política convergida.[EN] The present Thesis employs dynamic programming and reinforcement learning techniques in order to obtain optimal policies for controlling nonlinear systems with discrete and continuous states and actions. Initially, a review of the basic concepts of dynamic programming and reinforcement learning is carried out for systems with a finite number of states. After that, the extension of these techniques to systems with a large number of states or continuous state systems is analysed using approximation functions. The contributions of the Thesis are: -A combined identification/Q-function fitting methodology, which involves identification of a Takagi-Sugeno model, computation of (sub)optimal controllers from Linear Matrix Inequalities, and the subsequent data-based fitting of Q-function via monotonic optimisation. -A methodology for learning controllers using approximate dynamic programming via linear programming is presented. The methodology makes that ADP-LP approach can work in practical control applications with continuous state and input spaces. The proposed methodology estimates a lower bound and upper bound of the optimal value function through functional approximators. Guidelines are provided for data and regressor regularisation in order to obtain satisfactory results avoiding unbounded or ill-conditioned solutions. -A methodology of approximate dynamic programming via linear programming in order to obtain a better approximation of the optimal value function in a specific region of state space. The methodology proposes to gradually learn a policy using data available only in the exploration region. The exploration progressively increases the learning region until a converged policy is obtained.This work was supported by the National Department of Higher Education, Science, Technology and Innovation of Ecuador (SENESCYT), and the Spanish ministry of Economy and European Union, grant DPI2016-81002-R (AEI/FEDER,UE). The author also received the grant for a predoctoral stay, Programa de Becas Iberoamérica- Santander Investigación 2018, of the Santander Bank.Díaz Iza, HP. (2020). Value Function Estimation in Optimal Control via Takagi-Sugeno Models and Linear Programming [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139135TESI

Variance and Passivity Constrained Fuzzy Control for Nonlinear Ship Steering Systems with State Multiplicative Noises

The variance and passivity constrained fuzzy control problem for the nonlinear ship steering systems with state multiplicative noises is investigated. The continuous-time Takagi-Sugeno fuzzy model is used to represent the nonlinear ship steering systems with state multiplicative noises. In order to simultaneously achieve variance, passivity, and stability performances, some sufficient conditions are derived based on the Lyapunov theory. Employing the matrix transformation technique, these sufficient conditions can be expressed in terms of linear matrix inequalities. By solving the corresponding linear matrix inequality conditions, a parallel distributed compensation based fuzzy controller can be obtained to guarantee the stability of the closed-loop nonlinear ship steering systems subject to variance and passivity performance constraints. Finally, a numerical simulation example is provided to illustrate the usefulness and applicability of the proposed multiple performance constrained fuzzy control method

New Approaches in Automation and Robotics

The book New Approaches in Automation and Robotics offers in 22 chapters a collection of recent developments in automation, robotics as well as control theory. It is dedicated to researchers in science and industry, students, and practicing engineers, who wish to update and enhance their knowledge on modern methods and innovative applications. The authors and editor of this book wish to motivate people, especially under-graduate students, to get involved with the interesting field of robotics and mechatronics. We hope that the ideas and concepts presented in this book are useful for your own work and could contribute to problem solving in similar applications as well. It is clear, however, that the wide area of automation and robotics can only be highlighted at several spots but not completely covered by a single book

Contributions to fuzzy polynomial techniques for stability analysis and control

The present thesis employs fuzzy-polynomial control techniques in order to improve the stability analysis and control of nonlinear systems. Initially, it reviews the more extended techniques in the field of Takagi-Sugeno fuzzy systems, such as the more relevant results about polynomial and fuzzy polynomial systems. The basic framework uses fuzzy polynomial models by Taylor series and sum-of-squares techniques (semidefinite programming) in order to obtain stability guarantees. The contributions of the thesis are: ¿ Improved domain of attraction estimation of nonlinear systems for both continuous-time and discrete-time cases. An iterative methodology based on invariant-set results is presented for obtaining polynomial boundaries of such domain of attraction. ¿ Extension of the above problem to the case with bounded persistent disturbances acting. Different characterizations of inescapable sets with polynomial boundaries are determined. ¿ State estimation: extension of the previous results in literature to the case of fuzzy observers with polynomial gains, guaranteeing stability of the estimation error and inescapability in a subset of the zone where the model is valid. ¿ Proposal of a polynomial Lyapunov function with discrete delay in order to improve some polynomial control designs from literature. Preliminary extension to the fuzzy polynomial case. Last chapters present a preliminary experimental work in order to check and validate the theoretical results on real platforms in the future.Pitarch Pérez, JL. (2013). Contributions to fuzzy polynomial techniques for stability analysis and control [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/34773TESI

Discrete Time Systems

Discrete-Time Systems comprehend an important and broad research field. The consolidation of digital-based computational means in the present, pushes a technological tool into the field with a tremendous impact in areas like Control, Signal Processing, Communications, System Modelling and related Applications. This book attempts to give a scope in the wide area of Discrete-Time Systems. Their contents are grouped conveniently in sections according to significant areas, namely Filtering, Fixed and Adaptive Control Systems, Stability Problems and Miscellaneous Applications. We think that the contribution of the book enlarges the field of the Discrete-Time Systems with signification in the present state-of-the-art. Despite the vertiginous advance in the field, we also believe that the topics described here allow us also to look through some main tendencies in the next years in the research area

Measurements, Models, Systems and Design

531 s.

Recent advances on filtering and control for nonlinear stochastic complex systems with incomplete information: A survey

This Article is provided by the Brunel Open Access Publishing Fund - Copyright @ 2012 Hindawi PublishingSome recent advances on the filtering and control problems for nonlinear stochastic complex systems with incomplete information are surveyed. The incomplete information under consideration mainly includes missing measurements, randomly varying sensor delays, signal quantization, sensor saturations, and signal sampling. With such incomplete information, the developments on various filtering and control issues are reviewed in great detail. In particular, the addressed nonlinear stochastic complex systems are so comprehensive that they include conventional nonlinear stochastic systems, different kinds of complex networks, and a large class of sensor networks. The corresponding filtering and control technologies for such nonlinear stochastic complex systems are then discussed. Subsequently, some latest results on the filtering and control problems for the complex systems with incomplete information are given. Finally, conclusions are drawn and several possible future research directions are pointed out.This work was supported in part by the National Natural Science Foundation of China under Grant nos. 61134009, 61104125, 61028008, 61174136, 60974030, and 61074129, the Qing Lan Project of Jiangsu Province of China, the Project sponsored by SRF for ROCS of SEM of China, the Engineering and Physical Sciences Research Council EPSRC of the UK under Grant GR/S27658/01, the Royal Society of the UK, and the Alexander von Humboldt Foundation of Germany

Control of chaos in nonlinear circuits and systems

Nonlinear circuits and systems, such as electronic circuits (Chapter 5), power converters (Chapter 6), human brains (Chapter 7), phase lock loops (Chapter 8), sigma delta modulators (Chapter 9), etc, are found almost everywhere. Understanding nonlinear behaviours as well as control of these circuits and systems are important for real practical engineering applications. Control theories for linear circuits and systems are well developed and almost complete. However, different nonlinear circuits and systems could exhibit very different behaviours. Hence, it is difficult to unify a general control theory for general nonlinear circuits and systems. Up to now, control theories for nonlinear circuits and systems are still very limited. The objective of this book is to review the state of the art chaos control methods for some common nonlinear circuits and systems, such as those listed in the above, and stimulate further research and development in chaos control for nonlinear circuits and systems. This book consists of three parts. The first part of the book consists of reviews on general chaos control methods. In particular, a time-delayed approach written by H. Huang and G. Feng is reviewed in Chapter 1. A master slave synchronization problem for chaotic Lur’e systems is considered. A delay independent and delay dependent synchronization criteria are derived based on the H performance. The design of the time delayed feedback controller can be accomplished by means of the feasibility of linear matrix inequalities. In Chapter 2, a fuzzy model based approach written by H.K. Lam and F.H.F. Leung is reviewed. The synchronization of chaotic systems subject to parameter uncertainties is considered. A chaotic system is first represented by the fuzzy model. A switching controller is then employed to synchronize the systems. The stability conditions in terms of linear matrix inequalities are derived based on the Lyapunov stability theory. The tracking performance and parameter design of the controller are formulated as a generalized eigenvalue minimization problem which is solved numerically via some convex programming techniques. In Chapter 3, a sliding mode control approach written by Y. Feng and X. Yu is reviewed. Three kinds of sliding mode control methods, traditional sliding mode control, terminal sliding mode control and non-singular terminal sliding mode control, are employed for the control of a chaotic system to realize two different control objectives, namely to force the system states to converge to zero or to track desired trajectories. Observer based chaos synchronizations for chaotic systems with single nonlinearity and multi-nonlinearities are also presented. In Chapter 4, an optimal control approach written by C.Z. Wu, C.M. Liu, K.L. Teo and Q.X. Shao is reviewed. Systems with nonparametric regression with jump points are considered. The rough locations of all the possible jump points are identified using existing kernel methods. A smooth spline function is used to approximate each segment of the regression function. A time scaling transformation is derived so as to map the undecided jump points to fixed points. The approximation problem is formulated as an optimization problem and solved via existing optimization tools. The second part of the book consists of reviews on general chaos controls for continuous-time systems. In particular, chaos controls for Chua’s circuits written by L.A.B. Tôrres, L.A. Aguirre, R.M. Palhares and E.M.A.M. Mendes are discussed in Chapter 5. An inductorless Chua’s circuit realization is presented, as well as some practical issues, such as data analysis, mathematical modelling and dynamical characterization, are discussed. The tradeoff among the control objective, the control energy and the model complexity is derived. In Chapter 6, chaos controls for pulse width modulation current mode single phase H-bridge inverters written by B. Robert, M. Feki and H.H.C. Iu are discussed. A time delayed feedback controller is used in conjunction with the proportional controller in its simple form as well as in its extended form to stabilize the desired periodic orbit for larger values of the proportional controller gain. This method is very robust and easy to implement. In Chapter 7, chaos controls for epileptiform bursting in the brain written by M.W. Slutzky, P. Cvitanovic and D.J. Mogul are discussed. Chaos analysis and chaos control algorithms for manipulating the seizure like behaviour in a brain slice model are discussed. The techniques provide a nonlinear control pathway for terminating or potentially preventing epileptic seizures in the whole brain. The third part of the book consists of reviews on general chaos controls for discrete-time systems. In particular, chaos controls for phase lock loops written by A.M. Harb and B.A. Harb are discussed in Chapter 8. A nonlinear controller based on the theory of backstepping is designed so that the phase lock loops will not be out of lock. Also, the phase lock loops will not exhibit Hopf bifurcation and chaotic behaviours. In Chapter 9, chaos controls for sigma delta modulators written by B.W.K. Ling, C.Y.F. Ho and J.D. Reiss are discussed. A fuzzy impulsive control approach is employed for the control of the sigma delta modulators. The local stability criterion and the condition for the occurrence of limit cycle behaviours are derived. Based on the derived conditions, a fuzzy impulsive control law is formulated so that the occurrence of the limit cycle behaviours, the effect of the audio clicks and the distance between the state vectors and an invariant set are minimized supposing that the invariant set is nonempty. The state vectors can be bounded within any arbitrary nonempty region no matter what the input step size, the initial condition and the filter parameters are. The editors are much indebted to the editor of the World Scientific Series on Nonlinear Science, Prof. Leon Chua, and to Senior Editor Miss Lakshmi Narayan for their help and congenial processing of the edition