Adaptive sampled-data tracking for input constrained exothermic chemical reaction models

We consider digital input-constrained adaptive output feedback control of a class of nonlinear systems which arise as models for controlled exothermic chemical reactors.Our objective is set-point control of the temperature of the reaction, with prespecified asymptotic tracking accuracy set by the designer. Our approach is based on. Our objective is set-point control of the temperature of the reaction, with prespecified asymptotic tracking accuracy set by the designer. Our approach is based on lamda-tracking controllers, but we introduce a piecewise constant sampled-data output feedback strategy with adapted sampling period. The approach does not require any knowledge of the systems parameters, does not invoke an internal model, is simple in its design, copes with noise corrupted output measurements, and requires only a feasibility assumption in terms of the reference temperature and the input constraints

Adaptive and non-adaptive control without identification: a survey

Three related but distinct scenarios for tracking control of uncertain systems are reviewed: asymptotic tracking, approximate tracking with prescribed asymptotic error bound, tracking with prescribed transient behaviour. A variety of system classes are considered, ranging from finite-dimensional linear minimum-phase systems to nonlinear, infinite-dimensional systems described by functional differential equations. These classes are determined only by structural assumptions, such as stable zero dynamics and known relative degree. The objective is a single (and simple) control structure which is effective for every member of the underlying system class: no attempt is made to identify the particular system being controlled

An Investigation On Model Predictive Controllers’ Applications Of A Chemical Engineering Process

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2006Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2006Bu çalışmada, Model Öngörülü Kontrol edicilerin kimya mühendisliğinin en öenmli uygulamalarından biri olan kimyasal reaktörlerde kullanımı ve performansı incelenmiştir. Propilen oksit ve su reaksiyonundan ortaya çıkan propilen glikolun konsantrasyonu ve reaktörün sıcaklığı, propilen oksit ve soğutma suyunun akış debisinin ayarlanması ile control edilmiş, soğutma suyu sıcaklığı ve propilen oksitin giriş konsantrasyonu bozucu etki olarak değerlendirilmiştir. Tasarlanan kontrol edici, ayar noktası değişimleri ve bozucu etkilere karşı cevabı yönünden incelenmiştir. Bu etkilere kontrol edicinin kısa sürede cevap verdiği gözlemlenmiştir. Ayrıca kontrol edicinin gürbüzlüğünü test etmek amacı ile sistem parametreleri değiştirilmiş ve kontrol edicinin yeni sistemde de o modele ait olmayan basamak cevabı ile reaktörü kontrol edebildiği gözlemlenmiştir.In this study, applications and performance of MPC on chemical reactors, one of the most important chemical engineering applications, are examined. Propylene glycol concentration, resulting from reaction of propylene oxide and water, is controlled with reactor temperature by manipulating propylene oxide and coolant flow rates. Temperature of cooling water and initial concentration of propylene oxide are evaluated as measured disturbances. The designed controller is examined in terms of set point tracking and disturbance rejection. It is seen that the controller tracks the set point and rejects the effect of disturbances in a reasonably short time. Also in order to test the robustness of the controller, the system parameters have been changed and it is observed that the controller performs finely with old step response data model.Yüksek LisansM.Sc

Multi-rate data fusion for state and parameter estimation in (Bio-)chemical process engineering

For efficient operation, modern control approaches for biochemical process engineering require information on the states of the process such as temperature, humidity or chemical composition. Those measurement are gathered from a set of sensors which differ with respect to sampling rates and measurement quality. Furthermore, for biochemical processes in particular, analysis of physical samples is necessary, e.g., to infer cellular composition resulting in delayed information. As an alternative for the use of this delayed measurement for control, so-called soft-sensor approaches can be used to fuse delayed multirate measurements with the help of a mathematical process model and provide information on the current state of the process. In this manuscript we present a complete methodology based on cascaded unscented Kalman filters for state estimation from delayed and multi-rate measurements. The approach is demonstrated for two examples, an exothermic chemical reactor and a recently developed model for biopolymer production. The results indicate that the the current state of the systems can be accurately reconstructed and therefore represent a promising tool for further application in advanced model-based control not only of the considered processes but also of related processes

Malliprediktiivinen säädin Tennessee Eastman prosessille

This thesis aims to design a multivariable Model Predictive Control (MPC) scheme for a complex industrial process. The focus of the thesis is on the implementation and testing of a linear MPC control strategy combined with fault detection and diagnosis methods. The studied control methodology is based on a linear time invariant state-space model and the quadratic programming optimization procedure. The control scheme is realized as a supervisory one, where the MPC is used to calculate the optimal set point trajectories for the lower level PI controllers, thus aiming to decrease the fluctuations in the end product flows. The Tennessee Eastman (TE) process is used as the testing environment. The TE process is a benchmark based on a real process modified for testing. It has five units, four reactants, an inert, two products and a byproduct. The control objective is to maintain the production rate and the product quality at the desired level. To achieve this, the MPC implemented in this thesis gives setpoints to three stabilizing PI control loops around the reactor and the product stripper. The performance of the designed control systems is evaluated by inducing process disturbances, setpoint changes, and faults for two operational regimes. The obtained results show the efficiency of the adopted approach in handling disturbances and flexibility in control of different operational regimes without the need of retuning. To suppress the effects caused by faults, an additional level that provides fault detection and controller reconfiguration should be developed as further research.Tämän diplomityön tavoite on suunnitella monimuuttujainen-malliprediktiivinen säädin (MPC) teolliselle prosessille. Diplomityö keskittyy toteuttamaan ja testaamaan lineaarisen MPC strategian, joka yhdistettynä vikojen havainnointiin ja tunnistukseen sekä uudelleen konfigurointiin voidaan laajentaa vikasietoiseksi. Tutkittu säätöstrategia perustuu lineaariseen ajan suhteen muuttumattomaan tilataso-malliin ja neliöllisen ohjelmoinnin optimointimenetelmään. Säätö on toteutettu nk. ylemmän tason järjestelmänä, eli MPC:tä käytetään laskemaan optimaaliset asetusarvot alemman säätötason PI säätimille, tavoitteena vähentää vaihtelua lopputuotteen virroissa. Tennessee Eastman (TE) prosessia käytetään testiympäristönä. TE on testiprosessi, joka perustuu todelliseen teollisuuden prosessiin ja jota on muokattu testauskäyttöön sopivaksi. Prosessissa on viisi yksikköä, neljä lähtöainetta, inertti, kaksi tuotetta ja yksi sivutuote. Säätötavoite on ylläpitää haluttu taso tuotannon määrässä ja laadussa. Tämän saavuttamiseksi tässä diplomityössä toteutettu MPC antaa asetusarvoja kolmelle stabiloivalle PI-säätimelle reaktorin ja stripperin hallinnassa. Säätösysteemin suorituskykyä arvioitiin aiheuttamalla prosessiin häiriöitä, asetusarvon muutoksia ja vikoja eri operatiivisissa olosuhteissa. Saavutetut tulokset osoittavat valitun menetelmän tehokkuuden häiriöiden käsittelyyn ja joustavaan säätöön eri olosuhteissa. Tutkimuksen jatkokehityksenä vikojen vaikutuksen vaimentamiseksi säätöön tulisi lisätä taso, joka havaitsee viat ja uudelleen konfiguroi säätimen sen mukaisesti

모델기반강화학습을이용한공정제어및최적화

학위논문(박사)--서울대학교 대학원 :공과대학 화학생물공학부,2020. 2. 이종민.순차적 의사결정 문제는 공정 최적화의 핵심 분야 중 하나이다. 이 문제의 수치적 해법 중 가장 많이 사용되는 것은 순방향으로 작동하는 직접법 (direct optimization) 방법이지만, 몇가지 한계점을 지니고 있다. 최적해는 open-loop의 형태를 지니고 있으며, 불확정성이 존재할때 방법론의 수치적 복잡도가 증가한다는 것이다. 동적 계획법 (dynamic programming) 은 이러한 한계점을 근원적으로 해결할 수 있지만, 그동안 공정 최적화에 적극적으로 고려되지 않았던 이유는 동적 계획법의 결과로 얻어진 편미분 방정식 문제가 유한차원 벡터공간이 아닌 무한차원의 함수공간에서 다루어지기 때문이다. 소위 차원의 저주라고 불리는 이 문제를 해결하기 위한 한가지 방법으로서, 샘플을 이용한 근사적 해법에 초점을 둔 강화학습 방법론이 연구되어 왔다. 본 학위논문에서는 강화학습 방법론 중, 공정 최적화에 적합한 모델 기반 강화학습에 대해 연구하고, 이를 공정 최적화의 대표적인 세가지 순차적 의사결정 문제인 스케줄링, 상위단계 최적화, 하위단계 제어에 적용하는 것을 목표로 한다. 이 문제들은 각각 부분관측 마르코프 결정 과정 (partially observable Markov decision process), 제어-아핀 상태공간 모델 (control-affine state space model), 일반적 상태공간 모델 (general state space model)로 모델링된다. 또한 각 수치적 모델들을 해결하기 위해 point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP)로 불리는 방법들을 도입하였다. 이 세가지 문제와 방법론에서 제시된 특징들을 다음과 같이 요약할 수 있다: 첫번째로, 스케줄링 문제에서 closed-loop 피드백 형태의 해를 제시할 수 있었다. 이는 기존 직접법에서 얻을 수 없었던 형태로서, 강화학습의 강점을 부각할 수 있는 측면이라 생각할 수 있다. 두번째로 고려한 하위단계 제어 문제에서, 동적 계획법의 무한차원 함수공간 최적화 문제를 함수 근사 방법을 통해 유한차원 벡터공간 최적화 문제로 완화할 수 있는 방법을 도입하였다. 특히, 심층 신경망을 이용하여 함수 근사를 하였고, 이때 발생하는 여러가지 장점과 수렴 해석 결과를 본 학위논문에 실었다. 마지막 문제는 상위 단계 동적 최적화 문제이다. 동적 최적화 문제에서 발생하는 제약 조건하에서 강화학습을 수행하기 위해, 원-쌍대 미분동적 계획법 (primal-dual DDP) 방법론을 새로 제안하였다. 앞서 설명한 세가지 문제에 적용된 방법론을 검증하고, 동적 계획법이 직접법에 비견될 수 있는 방법론이라는 주장을 실증하기 위해 여러가지 공정 예제를 실었다.Sequential decision making problem is a crucial technology for plant-wide process optimization. While the dominant numerical method is the forward-in-time direct optimization, it is limited to the open-loop solution and has difficulty in considering the uncertainty. Dynamic programming method complements the limitations, nonetheless associated functional optimization suffers from the curse-of-dimensionality. The sample-based approach for approximating the dynamic programming, referred to as reinforcement learning (RL) can resolve the issue and investigated throughout this thesis. The method that accounts for the system model explicitly is in particular interest. The model-based RL is exploited to solve the three representative sequential decision making problems; scheduling, supervisory optimization, and regulatory control. The problems are formulated with partially observable Markov decision process, control-affine state space model, and general state space model, and associated model-based RL algorithms are point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP), respectively. The contribution for each problem can be written as follows: First, for the scheduling problem, we developed the closed-loop feedback scheme which highlights the strength compared to the direct optimization method. In the second case, the regulatory control problem is tackled by the function approximation method which relaxes the functional optimization to the finite dimensional vector space optimization. Deep neural networks (DNNs) is utilized as the approximator, and the advantages as well as the convergence analysis is performed in the thesis. Finally, for the supervisory optimization problem, we developed the novel constraint RL framework that uses the primal-dual DDP method. Various illustrative examples are demonstrated to validate the developed model-based RL algorithms and to support the thesis statement on which the dynamic programming method can be considered as a complementary method for direct optimization method.1. Introduction 1 1.1 Motivation and previous work 1 1.2 Statement of contributions 9 1.3 Outline of the thesis 11 2. Background and preliminaries 13 2.1 Optimization problem formulation and the principle of optimality 13 2.1.1 Markov decision process 15 2.1.2 State space model 19 2.2 Overview of the developed RL algorithms 28 2.2.1 Point based value iteration 28 2.2.2 Globalized dual heuristic programming 29 2.2.3 Differential dynamic programming 32 3. A POMDP framework for integrated scheduling of infrastructure maintenance and inspection 35 3.1 Introduction 35 3.2 POMDP solution algorithm 38 3.2.1 General point based value iteration 38 3.2.2 GapMin algorithm 46 3.2.3 Receding horizon POMDP 49 3.3 Problem formulation for infrastructure scheduling 54 3.3.1 State 56 3.3.2 Maintenance and inspection actions 57 3.3.3 State transition function 61 3.3.4 Cost function 67 3.3.5 Observation set and observation function 68 3.3.6 State augmentation 69 3.4 Illustrative example and simulation result 69 3.4.1 Structural point for the analysis of a high dimensional belief space 72 3.4.2 Infinite horizon policy under the natural deterioration process 72 3.4.3 Receding horizon POMDP 79 3.4.4 Validation of POMDP policy via Monte Carlo simulation 83 4. A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system 88 4.1 Introduction 88 4.2 Function approximation and learning with deep neural networks 91 4.2.1 GDHP with a function approximator 91 4.2.2 Stable learning of DNNs 96 4.2.3 Overall algorithm 103 4.3 Results and discussions 107 4.3.1 Example 1: Semi-batch reactor 107 4.3.2 Example 2: Diffusion-Convection-Reaction (DCR) process 120 5. Convergence analysis of the model-based deep reinforcement learning for optimal control of nonlinear control-affine system 126 5.1 Introduction 126 5.2 Convergence proof of globalized dual heuristic programming (GDHP) 128 5.3 Function approximation with deep neural networks 137 5.3.1 Function approximation and gradient descent learning 137 5.3.2 Forward and backward propagations of DNNs 139 5.4 Convergence analysis in the deep neural networks space 141 5.4.1 Lyapunov analysis of the neural network parameter errors 141 5.4.2 Lyapunov analysis of the closed-loop stability 150 5.4.3 Overall Lyapunov function 152 5.5 Simulation results and discussions 157 5.5.1 System description 158 5.5.2 Algorithmic settings 160 5.5.3 Control result 161 6. Primal-dual differential dynamic programming for constrained dynamic optimization of continuous system 170 6.1 Introduction 170 6.2 Primal-dual differential dynamic programming for constrained dynamic optimization 172 6.2.1 Augmented Lagrangian method 172 6.2.2 Primal-dual differential dynamic programming algorithm 175 6.2.3 Overall algorithm 179 6.3 Results and discussions 179 7. Concluding remarks 186 7.1 Summary of the contributions 187 7.2 Future works 189 Bibliography 192Docto