WH-EA: An Evolutionary Algorithm for Wiener-Hammerstein System Identification

[EN] Current methods to identify Wiener-Hammerstein systems using Best Linear Approximation (BLA) involve at least two steps. First, BLA is divided into obtaining front and back linear dynamics of the Wiener-Hammerstein model. Second, a re tting procedure of all parameters is carried out to reduce modelling errors. In this paper, a novel approach to identify Wiener-Hammerstein systems in a single step is proposed. is approach is based on a customized evolutionary algorithm (WH-EA) able to look for the best BLA split, capturing at the same time the process static nonlinearity with high precision. Furthermore, to correct possible errors in BLA estimation, the locations of poles and zeros are subtly modi ed within an adequate search space to allow a ne-tuning of the model. e performance of the proposed approach is analysed by using a demonstration example and a nonlinear system identi cation benchmark.This work was partially supported by the Spanish Ministry of Economy and Competitiveness (Project DPI2015-71443-R) and Salesian Polytechnic University of Ecuador through a Ph.D. scholarship granted to the first author.Zambrano-Abad, JC.; Sanchís Saez, J.; Herrero Durá, JM.; Martínez Iranzo, MA. (2018). WH-EA: An Evolutionary Algorithm for Wiener-Hammerstein System Identification. Complexity. 2018:1-17. https://doi.org/10.1155/2018/1753262S1172018Mora, L. A., & Amaya, J. E. (2017). Un Nuevo Método de Identificación Basado en la Respuesta Escalón en Lazo Abierto de Sistemas Sobre-amortiguados. Revista Iberoamericana de Automática e Informática Industrial RIAI, 14(1), 31-43. doi:10.1016/j.riai.2016.09.006Liu, T., Wang, Q.-G., & Huang, H.-P. (2013). A tutorial review on process identification from step or relay feedback test. Journal of Process Control, 23(10), 1597-1623. doi:10.1016/j.jprocont.2013.08.003Karnopp, D. C., Margolis, D. L., & Rosenberg, R. C. (2012). System Dynamics. doi:10.1002/9781118152812Bonilla, J., Roca, L., de la Calle, A., & Dormido, S. (2017). Modelo Dinámico de un Recuperador de Gases -Sales Fundidas para una Planta Termosolar Híbrida de Energías Renovables. Revista Iberoamericana de Automática e Informática Industrial RIAI, 14(1), 70-81. doi:10.1016/j.riai.2016.11.003Billings, S. A., & Fakhouri, S. Y. (1982). Identification of systems containing linear dynamic and static nonlinear elements. Automatica, 18(1), 15-26. doi:10.1016/0005-1098(82)90022-xLopes dos Santos, P., A. Ramos, J., & Martins de Carvalho, J. L. (2012). Identification of a Benchmark Wiener–Hammerstein: A bilinear and Hammerstein–Bilinear model approach. Control Engineering Practice, 20(11), 1156-1164. doi:10.1016/j.conengprac.2012.04.002Kalafatis, A., Arifin, N., Wang, L., & Cluett, W. R. (1995). A new approach to the identification of pH processes based on the Wiener model. Chemical Engineering Science, 50(23), 3693-3701. doi:10.1016/0009-2509(95)00214-pJurado, F. (2006). A method for the identification of solid oxide fuel cells using a Hammerstein model. Journal of Power Sources, 154(1), 145-152. doi:10.1016/j.jpowsour.2005.04.005Boubaker, S. (2017). Identification of nonlinear Hammerstein system using mixed integer-real coded particle swarm optimization: application to the electric daily peak-load forecasting. Nonlinear Dynamics, 90(2), 797-814. doi:10.1007/s11071-017-3693-9S Gaya, M. (2017). Estimation of Turbidity in Water Treatment Plant using Hammerstein-Wiener and Neural Network Technique. Indonesian Journal of Electrical Engineering and Computer Science, 5(3), 666. doi:10.11591/ijeecs.v5.i3.pp666-672Bai, E.-W., Cai, Z., Dudley-Javorosk, S., & Shields, R. K. (2009). Identification of a modified Wiener–Hammerstein system and its application in electrically stimulated paralyzed skeletal muscle modeling. Automatica, 45(3), 736-743. doi:10.1016/j.automatica.2008.09.023Haryanto, A., & Hong, K.-S. (2013). Maximum likelihood identification of Wiener–Hammerstein models. Mechanical Systems and Signal Processing, 41(1-2), 54-70. doi:10.1016/j.ymssp.2013.07.008Gómez, J. C., Jutan, A., & Baeyens, E. (2004). Wiener model identification and predictive control of a pH neutralisation process. IEE Proceedings - Control Theory and Applications, 151(3), 329-338. doi:10.1049/ip-cta:20040438Li, S., & Li, Y. (2016). Model predictive control of an intensified continuous reactor using a neural network Wiener model. Neurocomputing, 185, 93-104. doi:10.1016/j.neucom.2015.12.048Zhang, Q., Wang, Q., & Li, G. (2016). Nonlinear modeling and predictive functional control of Hammerstein system with application to the turntable servo system. Mechanical Systems and Signal Processing, 72-73, 383-394. doi:10.1016/j.ymssp.2015.09.011Ławryńczuk, M. (2016). Nonlinear predictive control of dynamic systems represented by Wiener–Hammerstein models. Nonlinear Dynamics, 86(2), 1193-1214. doi:10.1007/s11071-016-2957-0Schoukens, M., Pintelon, R., & Rolain, Y. (2014). Identification of Wiener–Hammerstein systems by a nonparametric separation of the best linear approximation. Automatica, 50(2), 628-634. doi:10.1016/j.automatica.2013.12.027Vanbeylen, L. (2014). A fractional approach to identify Wiener–Hammerstein systems. Automatica, 50(3), 903-909. doi:10.1016/j.automatica.2013.12.013Sjöberg, J., Lauwers, L., & Schoukens, J. (2012). Identification of Wiener–Hammerstein models: Two algorithms based on the best split of a linear model applied to the SYSID’09 benchmark problem. Control Engineering Practice, 20(11), 1119-1125. doi:10.1016/j.conengprac.2012.07.001Westwick, D. T., & Schoukens, J. (2012). Initial estimates of the linear subsystems of Wiener–Hammerstein models. Automatica, 48(11), 2931-2936. doi:10.1016/j.automatica.2012.06.091Tan, A. H., Wong, H. K., & Godfrey, K. (2012). Identification of a Wiener–Hammerstein system using an incremental nonlinear optimisation technique. Control Engineering Practice, 20(11), 1140-1148. doi:10.1016/j.conengprac.2012.04.007Naitali, A., & Giri, F. (2015). Wiener–Hammerstein system identification – an evolutionary approach. International Journal of Systems Science, 47(1), 45-61. doi:10.1080/00207721.2015.1027758Schoukens, J., Lataire, J., Pintelon, R., Vandersteen, G., & Dobrowiecki, T. (2009). Robustness Issues of the Best Linear Approximation of a Nonlinear System. IEEE Transactions on Instrumentation and Measurement, 58(5), 1737-1745. doi:10.1109/tim.2009.2012948Ljung, L., & Singh, R. (2012). Version 8 of the Matlab System Identification Toolbox. IFAC Proceedings Volumes, 45(16), 1826-1831. doi:10.3182/20120711-3-be-2027.0006

Identification of nonlinear processes based on Wiener-Hammerstein models and heuristic optimization.

[ES] En muchos campos de la ingeniería los modelos matemáticos son utilizados para describir el comportamiento de los sistemas, procesos o fenómenos. Hoy en día, existen varias técnicas o métodos que pueden ser usadas para obtener estos modelos. Debido a su versatilidad y simplicidad, a menudo se prefieren los métodos de identificación de sistemas. Por lo general, estos métodos requieren la definición de una estructura y la estimación computacional de los parámetros que la componen utilizando un conjunto de procedimientos y mediciones de las señales de entrada y salida del sistema. En el contexto de la identificación de sistemas no lineales, un desafío importante es la selección de la estructura. En el caso de que el sistema a identificar presente una no linealidad de tipo estático, los modelos orientados a bloques, pueden ser útiles para definir adecuadamente una estructura. Sin embargo, el diseñador puede enfrentarse a cierto grado de incertidumbre al seleccionar el modelo orientado a bloques adecuado en concordancia con el sistema real. Además de este inconveniente, se debe tener en cuenta que la estimación de algunos modelos orientados a bloques no es sencilla, como es el caso de los modelos de Wiener-Hammerstein que consisten en un bloque NL en medio de dos subsistemas LTI. La presencia de dos subsistemas LTI en los modelos de Wiener-Hammerstein es lo que principalmente dificulta su estimación. Generalmente, el procedimiento de identificación comienza con la estimación de la dinámica lineal, y el principal desafío es dividir esta dinámica entre los dos bloques LTI. Por lo general, esto implica una alta interacción del usuario para desarrollar varios procedimientos, y el modelo final estimado depende principalmente de estas etapas previas. El objetivo de esta tesis es contribuir a la identificación de los modelos de Wiener-Hammerstein. Esta contribución se basa en la presentación de dos nuevos algoritmos para atender aspectos específicos que no han sido abordados en la identificación de este tipo de modelos. El primer algoritmo, denominado WH-EA, permite estimar todos los parámetros de un modelo de Wiener-Hammerstein con un solo procedimiento a partir de un modelo dinámico lineal. Con WH-EA, una buena estimación no depende de procedimientos intermedios ya que el algoritmo evolutivo simultáneamente busca la mejor distribución de la dinámica, ajusta con precisión la ubicación de los polos y los ceros y captura la no linealidad estática. Otra ventaja importante de este algoritmo es que bajo consideraciones específicas y utilizando una señal de excitación adecuada, es posible crear un enfoque unificado que permite también la identificación de los modelos de Wiener y Hammerstein, que son casos particulares del modelo de Wiener-Hammerstein cuando uno de sus bloques LTI carece de dinámica. Lo interesante de este enfoque unificado es que con un mismo algoritmo es posible identificar los modelos de Wiener, Hammerstein y Wiener-Hammerstein sin que el usuario especifique de antemano el tipo de estructura a identificar. El segundo algoritmo llamado WH-MOEA, permite abordar el problema de identificación como un Problema de Optimización Multiobjetivo (MOOP). Sobre la base de este algoritmo se presenta un nuevo enfoque para la identificación de los modelos de Wiener-Hammerstein considerando un compromiso entre la precisión alcanzada y la complejidad del modelo. Con este enfoque es posible comparar varios modelos con diferentes prestaciones incluyendo como un objetivo de identificación el número de parámetros que puede tener el modelo estimado. El aporte de este enfoque se sustenta en el hecho de que en muchos problemas de ingeniería los requisitos de diseño y las preferencias del usuario no siempre apuntan a la precisión del modelo como un único objetivo, sino que muchas veces la complejidad es también un factor predominante en la toma de decisiones.[CA] En molts camps de l'enginyeria els models matemàtics són utilitzats per a descriure el comportament dels sistemes, processos o fenòmens. Hui dia, existeixen diverses tècniques o mètodes que poden ser usades per a obtindre aquests models. A causa de la seua versatilitat i simplicitat, sovint es prefereixen els mètodes d'identificació de sistemes. En general, aquests mètodes requereixen la definició d'una estructura i l'estimació computacional dels paràmetres que la componen utilitzant un conjunt de procediments i mesuraments dels senyals d'entrada i eixida del sistema. En el context de la identificació de sistemes no lineals, un desafiament important és la selecció de l'estructura. En el cas que el sistema a identificar presente una no linealitat de tipus estàtic, els models orientats a blocs, poden ser útils per a definir adequadament una estructura. No obstant això, el dissenyador pot enfrontar-se a cert grau d'incertesa en seleccionar el model orientat a blocs adequat en concordança amb el sistema real. A més d'aquest inconvenient, s'ha de tindre en compte que l'estimació d'alguns models orientats a blocs no és senzilla, com és el cas dels models de Wiener-Hammerstein que consisteixen en un bloc NL enmig de dos subsistemes LTI. La presència de dos subsistemes LTI en els models de Wiener-Hammerstein és el que principalment dificulta la seua estimació. Generalment, el procediment d'identificació comença amb l'estimació de la dinàmica lineal, i el principal desafiament és dividir aquesta dinàmica entre els dos blocs LTI. En general, això implica una alta interacció de l'usuari per a desenvolupar diversos procediments, i el model final estimat depén principalment d'aquestes etapes prèvies. L'objectiu d'aquesta tesi és contribuir a la identificació dels models de Wiener-Hammerstein. Aquesta contribució es basa en la presentació de dos nous algorismes per a atendre aspectes específics que no han sigut adreçats en la identificació d'aquesta mena de models. El primer algorisme, denominat WH-EA (Algorisme Evolutiu per a la identificació de sistemes de Wiener-Hammerstein), permet estimar tots els paràmetres d'un model de Wiener-Hammerstein amb un sol procediment a partir d'un model dinàmic lineal. Amb WH-EA, una bona estimació no depén de procediments intermedis ja que l'algorisme evolutiu simultàniament busca la millor distribució de la dinàmica, afina la ubicació dels pols i els zeros i captura la no linealitat estàtica. Un altre avantatge important d'aquest algorisme és que sota consideracions específiques i utilitzant un senyal d'excitació adequada, és possible crear un enfocament unificat que permet també la identificació dels models de Wiener i Hammerstein, que són casos particulars del model de Wiener-Hammerstein quan un dels seus blocs LTI manca de dinàmica. L'interessant d'aquest enfocament unificat és que amb un mateix algorisme és possible identificar els models de Wiener, Hammerstein i Wiener-Hammerstein sense que l'usuari especifique per endavant el tipus d'estructura a identificar. El segon algorisme anomenat WH-MOEA (Algorisme evolutiu multi-objectiu per a la identificació de models de Wiener-Hammerstein), permet abordar el problema d'identificació com un Problema d'Optimització Multiobjectiu (MOOP). Sobre la base d'aquest algorisme es presenta un nou enfocament per a la identificació dels models de Wiener-Hammerstein considerant un compromís entre la precisió aconseguida i la complexitat del model. Amb aquest enfocament és possible comparar diversos models amb diferents prestacions incloent com un objectiu d'identificació el nombre de paràmetres que pot tindre el model estimat. L'aportació d'aquest enfocament se sustenta en el fet que en molts problemes d'enginyeria els requisits de disseny i les preferències de l'usuari no sempre apunten a la precisió del model com un únic objectiu, sinó que moltes vegades la complexitat és també un factor predominant en la presa de decisions.[EN] In several engineering fields, mathematical models are used to describe the behaviour of systems, processes or phenomena. Nowadays, there are several techniques or methods for obtaining mathematical models. Because of their versatility and simplicity, system identification methods are often preferred. Generally, systems identification methods require defining a structure and estimating computationally the parameters that make it up, using a set of procedures y measurements of the system's input and output signals. In the context of nonlinear system identification, a significant challenge is the structure selection. In the case that the system to be identified presents a static type of nonlinearity, block-oriented models can be useful to define a suitable structure. However, the designer may face a certain degree of uncertainty when selecting the block-oriented model in accordance with the real system. In addition to this inconvenience, the estimation of some block-oriented models is not an easy task, as is the case with the Wiener-Hammerstein models consisting of a NL block in the middle of two LTI subsystems. The presence of two LTI subsystems in the Wiener-Hammerstein models is what mainly makes their estimation difficult. Generally, the identification procedure begins with the estimation of the linear dynamics, and the main challenge is to split this dynamic between the two LTI block. Usually, this implies a high user interaction to develop several procedures, and the final model estimated mostly depends on these previous stages. The aim of this thesis is to contribute to the identification of the Wiener-Hammerstein models. This contribution is based on the presentation of two new algorithms to address specific aspects that have not been addressed in the identification of this type of model. The first algorithm, called WH-EA (An Evolutionary Algorithm for Wiener-Hammerstein System Identification), allows estimating all the parameters of a Wiener-Hammerstein model with a single procedure from a linear dynamic model. With WH-EA, a good estimate does not depend on intermediate procedures since the evolutionary algorithm looks for the best dynamic division, while the locations of the poles and zeros are fine-tuned, and nonlinearity is captured simultaneously. Another significant advantage of this algorithm is that under specific considerations and using a suitable excitation signal; it is possible to create a unified approach that also allows the identification of Wiener and Hammerstein models which are particular cases of the Wiener-Hammerstein model when one of its LTI blocks lacks dynamics. What is interesting about this unified approach is that with the same algorithm, it is possible to identify Wiener, Hammerstein, and Wiener-Hammerstein models without the user specifying in advance the type of structure to be identified. The second algorithm called WH-MOEA (Multi-objective Evolutionary Algorithm for Wiener-Hammerstein identification), allows to address the identification problem as a Multi-Objective Optimisation Problem (MOOP). Based on this algorithm, a new approach for the identification of Wiener-Hammerstein models is presented considering a compromise between the accuracy achieved and the model complexity. With this approach, it is possible to compare several models with different performances, including as an identification target the number of parameters that the estimated model may have. The contribution of this approach is based on the fact that in many engineering problems the design requirements and user's preferences do not always point to the accuracy of the model as a single objective, but many times the complexity is also a predominant factor in decision-making.Zambrano Abad, JC. (2021). Identification of nonlinear processes based on Wiener-Hammerstein models and heuristic optimization [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/171739TESI

WH-MOEA: A Multi-Objective Evolutionary Algorithm for Wiener-Hammerstein System Identification. A Novel Approach for Trade-Off Analysis Between Complexity and Accuracy

[EN] Several approaches have been presented to identify Wiener-Hammerstein models, most of them starting from a linear dynamic model whose poles and zeros are distributed around the static non- linearity. To achieve good precision in the estimation, the Best Linear Approximation (BLA) has usually been used to represent the linear dynamics, while static non-linearity has been arbitrarily parameterised without considering model complexity. In this paper, identification of Wiener, Hammerstein or Wiener-Hammerstein models is stated as a multiobjective optimisation problem (MOP), with a trade-off between accuracy and model complexity. Precision is quantified with the Mean-Absolute-Error (MAE) between the real and estimated output, while complexity is based on the number of poles, zeros and points of the static non- linearity. To solve the MOP, WH-MOEA, a new multiobjective evolutionary algorithm (MOEA) is proposed. From a linear structure, WH-MOEA will generate a set of optimal models considering a static non-linearity with a variable number of points. Using WH-MOEA, a procedure is also proposed to analyse various linear structures with different numbers of poles and zeros (known as design concepts). A comparison of the Pareto fronts of each design concept allows a more in-depth analysis to select the most appropriate model according to the user¿s needs. Finally, a complex numerical example and a real thermal process based on a Peltier cell are identified, showing the procedure¿s goodness. The results show that it can be useful to consider the simultaneously precision and complexity of a block-oriented model (Wiener, Hammerstein or Wiener- Hammerstein) in a non-linear process identification.This work was supported in part by the Ministerio de Ciencia, Innovación y Universidades, Spain, under Grant RTI2018-096904-B-I00-AR, and in part by the Salesian Polytechnic University of Ecuador through a Ph.D. scholarships granted to J. Zambrano.Zambrano, J.; Sanchís Saez, J.; Herrero Durá, JM.; Martínez Iranzo, MA. (2020). WH-MOEA: A Multi-Objective Evolutionary Algorithm for Wiener-Hammerstein System Identification. A Novel Approach for Trade-Off Analysis Between Complexity and Accuracy. IEEE Access. 8:228655-228674. https://doi.org/10.1109/ACCESS.2020.3046352228655228674

Fast identification of Wiener-Hammerstein systems using discrete optimisation

A fast identification algorithm for Wiener-Hammerstein systems is proposed. The computational cost of separating the front and the back linear time-invariant (LTI) block dynamics is significantly improved by using discrete optimisation. The discrete optimisation is implemented as a genetic algorithm. Numerical results confirm the efficiency and accuracy of the proposed approach

Nonlinear state-space identification using deep encoder networks

Nonlinear state-space identification for dynamical systems is most often performed by minimizing the simulation error to reduce the effect of model errors. This optimization problem becomes computationally expensive for large datasets. Moreover, the problem is also strongly non-convex, often leading to sub-optimal parameter estimates. This paper introduces a method that approximates the simulation loss by splitting the data set into multiple independent sections similar to the multiple shooting method. This splitting operation allows for the use of stochastic gradient optimization methods which scale well with data set size and has a smoothing effect on the non-convex cost function. The main contribution of this paper is the introduction of an encoder function to estimate the initial state at the start of each section. The encoder function estimates the initial states using a feed-forward neural network starting from historical input and output samples. The efficiency and performance of the proposed state-space encoder method is illustrated on two well-known benchmarks where, for instance, the method achieves the lowest known simulation error on the Wiener--Hammerstein benchmark

On evolutionary system identification with applications to nonlinear benchmarks

This paper presents a record of the participation of the authors in a workshop on nonlinear system identification held in 2016. It provides a summary of a keynote lecture by one of the authors and also gives an account of how the authors developed identification strategies and methods for a number of benchmark nonlinear systems presented as challenges, before and during the workshop. It is argued here that more general frameworks are now emerging for nonlinear system identification, which are capable of addressing substantial ranges of problems. One of these frameworks is based on evolutionary optimisation (EO); it is a framework developed by the authors in previous papers and extended here. As one might expect from the ‘no-free-lunch’ theorem for optimisation, the methodology is not particularly sensitive to the particular (EO) algorithm used, and a number of different variants are presented in this paper, some used for the first time in system identification problems, which show equal capability. In fact, the EO approach advocated in this paper succeeded in finding the best solutions to two of the three benchmark problems which motivated the workshop. The paper provides considerable discussion on the approaches used and makes a number of suggestions regarding best practice; one of the major new opportunities identified here concerns the application of grey-box models which combine the insight of any prior physical-law based models (white box) with the power of machine learners with universal approximation properties (black box)

Deep Subspace Encoders for Nonlinear System Identification

Using Artificial Neural Networks (ANN) for nonlinear system identification has proven to be a promising approach, but despite of all recent research efforts, many practical and theoretical problems still remain open. Specifically, noise handling and models, issues of consistency and reliable estimation under minimisation of the prediction error are the most severe problems. The latter comes with numerous practical challenges such as explosion of the computational cost in terms of the number of data samples and the occurrence of instabilities during optimization. In this paper, we aim to overcome these issues by proposing a method which uses a truncated prediction loss and a subspace encoder for state estimation. The truncated prediction loss is computed by selecting multiple truncated subsections from the time series and computing the average prediction loss. To obtain a computationally efficient estimation method that minimizes the truncated prediction loss, a subspace encoder represented by an artificial neural network is introduced. This encoder aims to approximate the state reconstructability map of the estimated model to provide an initial state for each truncated subsection given past inputs and outputs. By theoretical analysis, we show that, under mild conditions, the proposed method is locally consistent, increases optimization stability, and achieves increased data efficiency by allowing for overlap between the subsections. Lastly, we provide practical insights and user guidelines employing a numerical example and state-of-the-art benchmark results.Comment: Accepted in Automatic

Identification of Nonlinear Systems From the Knowledge Around Different Operating Conditions: A Feed-Forward Multi-Layer ANN Based Approach

The paper investigates nonlinear system identification using system output data at various linearized operating points. A feed-forward multi-layer Artificial Neural Network (ANN) based approach is used for this purpose and tested for two target applications i.e. nuclear reactor power level monitoring and an AC servo position control system. Various configurations of ANN using different activation functions, number of hidden layers and neurons in each layer are trained and tested to find out the best configuration. The training is carried out multiple times to check for consistency and the mean and standard deviation of the root mean square errors (RMSE) are reported for each configuration.Comment: "6 pages, 9 figures; The Second IEEE International Conference on Parallel, Distributed and Grid Computing (PDGC-2012), December 2012, Solan