Development of a computationally efficient model for the control of Ziegler-Natta catalysed industrial production of high density polyethylene

High density polyethylene is commonly produced by the slurry phase co-polymerisation of ethylene and other alkenes, using heterogeneous titanium-based Ziegler-Natta catalysts. During grade transitions, when reactor conditions are manipulated to change polymer properties, significant quantities of off-specification product result. Implementing a model-predictive controller based on a dynamic reactor model may allow for minimising losses during unsteady-state operation. Such a model must be developed from a fundamental understanding of polymerisation reaction kinetics and the interaction of effects at various scales, including those of catalyst sites, catalyst/polymer particles and reactor hydrodynamics. The model must also be computationally efficient enough for application to real-time control. The recently-developed pseudo-sites model was used as a fundamental kinetic explanation of polymer property distributions and catalyst activity profiles, in contrast to empirical multi-site models. Laboratory polymerisation experiments were performed at industrially-relevant conditions. Kinetic parameters were fitted to the data, using a novel proposed regression procedure to extract meaningful kinetic parameters. A dynamic reactor model was developed, based on the Segregation Approach. Whereas the more common Population Balance Model must consider multivariate distributions of population members within a chosen volume and requires partial differential equation solution, the Segregation Approach can generate the moments of a distribution by evaluating the evolution of properties without requiring solution over the whole volume. The Segregation Approach and PBM were rigorously compared in the context of Particle Size Distributions, and the Segregation Approach shown to be an order of magnitude more computationally efficient. Steady-state industrial data was used to reconcile model predictions for laboratory and industrial polymerisation. This was the first application of the pseudo-sites model to laboratory data, and first extension to industrial scale. Unsteady-state data from three industrial grade transitions was used to validate the reactor model, which closely matched industrial reactor performance. The model simulated 30-40 hours of real time in 15-25 seconds of calculation time. The reactor model was used to propose improved grade transition strategies; transition duration and waste production were improved by 20-40%. The reactor model has been shown to accurately reproduce real-world results, and is computationally efficient enough to be applied to model-based control applications

모델기반강화학습을이용한공정제어및최적화

학위논문(박사)--서울대학교 대학원 :공과대학 화학생물공학부,2020. 2. 이종민.순차적 의사결정 문제는 공정 최적화의 핵심 분야 중 하나이다. 이 문제의 수치적 해법 중 가장 많이 사용되는 것은 순방향으로 작동하는 직접법 (direct optimization) 방법이지만, 몇가지 한계점을 지니고 있다. 최적해는 open-loop의 형태를 지니고 있으며, 불확정성이 존재할때 방법론의 수치적 복잡도가 증가한다는 것이다. 동적 계획법 (dynamic programming) 은 이러한 한계점을 근원적으로 해결할 수 있지만, 그동안 공정 최적화에 적극적으로 고려되지 않았던 이유는 동적 계획법의 결과로 얻어진 편미분 방정식 문제가 유한차원 벡터공간이 아닌 무한차원의 함수공간에서 다루어지기 때문이다. 소위 차원의 저주라고 불리는 이 문제를 해결하기 위한 한가지 방법으로서, 샘플을 이용한 근사적 해법에 초점을 둔 강화학습 방법론이 연구되어 왔다. 본 학위논문에서는 강화학습 방법론 중, 공정 최적화에 적합한 모델 기반 강화학습에 대해 연구하고, 이를 공정 최적화의 대표적인 세가지 순차적 의사결정 문제인 스케줄링, 상위단계 최적화, 하위단계 제어에 적용하는 것을 목표로 한다. 이 문제들은 각각 부분관측 마르코프 결정 과정 (partially observable Markov decision process), 제어-아핀 상태공간 모델 (control-affine state space model), 일반적 상태공간 모델 (general state space model)로 모델링된다. 또한 각 수치적 모델들을 해결하기 위해 point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP)로 불리는 방법들을 도입하였다. 이 세가지 문제와 방법론에서 제시된 특징들을 다음과 같이 요약할 수 있다: 첫번째로, 스케줄링 문제에서 closed-loop 피드백 형태의 해를 제시할 수 있었다. 이는 기존 직접법에서 얻을 수 없었던 형태로서, 강화학습의 강점을 부각할 수 있는 측면이라 생각할 수 있다. 두번째로 고려한 하위단계 제어 문제에서, 동적 계획법의 무한차원 함수공간 최적화 문제를 함수 근사 방법을 통해 유한차원 벡터공간 최적화 문제로 완화할 수 있는 방법을 도입하였다. 특히, 심층 신경망을 이용하여 함수 근사를 하였고, 이때 발생하는 여러가지 장점과 수렴 해석 결과를 본 학위논문에 실었다. 마지막 문제는 상위 단계 동적 최적화 문제이다. 동적 최적화 문제에서 발생하는 제약 조건하에서 강화학습을 수행하기 위해, 원-쌍대 미분동적 계획법 (primal-dual DDP) 방법론을 새로 제안하였다. 앞서 설명한 세가지 문제에 적용된 방법론을 검증하고, 동적 계획법이 직접법에 비견될 수 있는 방법론이라는 주장을 실증하기 위해 여러가지 공정 예제를 실었다.Sequential decision making problem is a crucial technology for plant-wide process optimization. While the dominant numerical method is the forward-in-time direct optimization, it is limited to the open-loop solution and has difficulty in considering the uncertainty. Dynamic programming method complements the limitations, nonetheless associated functional optimization suffers from the curse-of-dimensionality. The sample-based approach for approximating the dynamic programming, referred to as reinforcement learning (RL) can resolve the issue and investigated throughout this thesis. The method that accounts for the system model explicitly is in particular interest. The model-based RL is exploited to solve the three representative sequential decision making problems; scheduling, supervisory optimization, and regulatory control. The problems are formulated with partially observable Markov decision process, control-affine state space model, and general state space model, and associated model-based RL algorithms are point based value iteration (PBVI), globalized dual heuristic programming (GDHP), and differential dynamic programming (DDP), respectively. The contribution for each problem can be written as follows: First, for the scheduling problem, we developed the closed-loop feedback scheme which highlights the strength compared to the direct optimization method. In the second case, the regulatory control problem is tackled by the function approximation method which relaxes the functional optimization to the finite dimensional vector space optimization. Deep neural networks (DNNs) is utilized as the approximator, and the advantages as well as the convergence analysis is performed in the thesis. Finally, for the supervisory optimization problem, we developed the novel constraint RL framework that uses the primal-dual DDP method. Various illustrative examples are demonstrated to validate the developed model-based RL algorithms and to support the thesis statement on which the dynamic programming method can be considered as a complementary method for direct optimization method.1. Introduction 1 1.1 Motivation and previous work 1 1.2 Statement of contributions 9 1.3 Outline of the thesis 11 2. Background and preliminaries 13 2.1 Optimization problem formulation and the principle of optimality 13 2.1.1 Markov decision process 15 2.1.2 State space model 19 2.2 Overview of the developed RL algorithms 28 2.2.1 Point based value iteration 28 2.2.2 Globalized dual heuristic programming 29 2.2.3 Differential dynamic programming 32 3. A POMDP framework for integrated scheduling of infrastructure maintenance and inspection 35 3.1 Introduction 35 3.2 POMDP solution algorithm 38 3.2.1 General point based value iteration 38 3.2.2 GapMin algorithm 46 3.2.3 Receding horizon POMDP 49 3.3 Problem formulation for infrastructure scheduling 54 3.3.1 State 56 3.3.2 Maintenance and inspection actions 57 3.3.3 State transition function 61 3.3.4 Cost function 67 3.3.5 Observation set and observation function 68 3.3.6 State augmentation 69 3.4 Illustrative example and simulation result 69 3.4.1 Structural point for the analysis of a high dimensional belief space 72 3.4.2 Infinite horizon policy under the natural deterioration process 72 3.4.3 Receding horizon POMDP 79 3.4.4 Validation of POMDP policy via Monte Carlo simulation 83 4. A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system 88 4.1 Introduction 88 4.2 Function approximation and learning with deep neural networks 91 4.2.1 GDHP with a function approximator 91 4.2.2 Stable learning of DNNs 96 4.2.3 Overall algorithm 103 4.3 Results and discussions 107 4.3.1 Example 1: Semi-batch reactor 107 4.3.2 Example 2: Diffusion-Convection-Reaction (DCR) process 120 5. Convergence analysis of the model-based deep reinforcement learning for optimal control of nonlinear control-affine system 126 5.1 Introduction 126 5.2 Convergence proof of globalized dual heuristic programming (GDHP) 128 5.3 Function approximation with deep neural networks 137 5.3.1 Function approximation and gradient descent learning 137 5.3.2 Forward and backward propagations of DNNs 139 5.4 Convergence analysis in the deep neural networks space 141 5.4.1 Lyapunov analysis of the neural network parameter errors 141 5.4.2 Lyapunov analysis of the closed-loop stability 150 5.4.3 Overall Lyapunov function 152 5.5 Simulation results and discussions 157 5.5.1 System description 158 5.5.2 Algorithmic settings 160 5.5.3 Control result 161 6. Primal-dual differential dynamic programming for constrained dynamic optimization of continuous system 170 6.1 Introduction 170 6.2 Primal-dual differential dynamic programming for constrained dynamic optimization 172 6.2.1 Augmented Lagrangian method 172 6.2.2 Primal-dual differential dynamic programming algorithm 175 6.2.3 Overall algorithm 179 6.3 Results and discussions 179 7. Concluding remarks 186 7.1 Summary of the contributions 187 7.2 Future works 189 Bibliography 192Docto

Ono: an open platform for social robotics

In recent times, the focal point of research in robotics has shifted from industrial ro- bots toward robots that interact with humans in an intuitive and safe manner. This evolution has resulted in the subfield of social robotics, which pertains to robots that function in a human environment and that can communicate with humans in an int- uitive way, e.g. with facial expressions. Social robots have the potential to impact many different aspects of our lives, but one particularly promising application is the use of robots in therapy, such as the treatment of children with autism. Unfortunately, many of the existing social robots are neither suited for practical use in therapy nor for large scale studies, mainly because they are expensive, one-of-a-kind robots that are hard to modify to suit a specific need. We created Ono, a social robotics platform, to tackle these issues. Ono is composed entirely from off-the-shelf components and cheap materials, and can be built at a local FabLab at the fraction of the cost of other robots. Ono is also entirely open source and the modular design further encourages modification and reuse of parts of the platform

Metal-Organic Frameworks in Germany: from Synthesis to Function

Metal-organic frameworks (MOFs) are constructed from a combination of inorganic and organic units to produce materials which display high porosity, among other unique and exciting properties. MOFs have shown promise in many wide-ranging applications, such as catalysis and gas separations. In this review, we highlight MOF research conducted by Germany-based research groups. Specifically, we feature approaches for the synthesis of new MOFs, high-throughput MOF production, advanced characterization methods and examples of advanced functions and properties

Synthesis of bio-functional nanomaterials in reactive plasma discharges

Plasma processing technologies have been extensively used as surface modification platforms in many biomedical applications. Particularly, plasma polymerization (PP) is a versatile deposition technology which has the potential to deliver biocompatible interfaces for a myriad of medical devices. To successfully translate new materials for specific clinical applications, the plasma process needs to be scalable and incorporate appropriate control feedback strategies. However, the plasma medium in PP is exceptionally complex and identifying the main physical quantities that allow a suitable formulation and description of the interface growth mechanisms is challenging. The first part of the thesis reports the design and optimization of a single step ion assisted PP process to create plasma-activated coatings (PAC) that meet the extreme mechanical demands for cardiovascular implants and in particular stents. An ideal working window in the parameter space is identified, and found suitable for the synthesis of PAC interfaces that are mechanically robust, hemocompatibility and allow one step covalent protein immobilization without the need for chemical processes. This window is identified by combining plasma optical emission spectroscopy (OES) with a comprehensive macroscopic process description that isolates key coating growth mechanisms. During process scalability, OES diagnostics revealed the formation of plasma polymer nanoparticles (nanoP3), usually known as plasma dust, in parallel with the deposition of PAC coatings. The second part of the thesis reports the demonstration of carbonaceous plasma nanoparticles for nanomedicine applications. By controlling nanoparticle formation and collection, nanoP3 were engineered with unique immobilization capabilities facilitating multifunctional nanocarriers. The unique surface chemistry of nanoP3, allowing a robust immobilization of the cargo without the need for intermediate functionalization strategies, has great potential to overcome major limitations of currently proposed platforms. As many of the favorable characteristics of nanoP3 are inherent to the fabrication process, this work proposes PP as a nanoparticle synthesis route with valuable potential for broad clinical and commercial applications

Polymer Reactor Modeling, Design and Monitoring

Polymers range from synthetic plastics, such as polyacrylates, to natural biopolymers, such as proteins and DNA. The large molecular mass of polymers and our ability to manipulate their compositions and molecular structures have allowed for producing synthetic polymers with attractive properties. new polymers with remarkable characteristics are synthesized. Because of the huge production volume of commodity polymers, a little improvement in the operation of commodity-polymer processes can lead to significant economic gains. On the other hand, a little improvement in the quality of specialty polymers can lead to substantial increase in economic profits

Barriers to success: A technical review on the limits and possible future roles of small scale gasifiers

Literature and manuals refer to biomass gasification as one of the most efficient processes for power generation, highlighting features, such as residual biomass use, distributed generation and carbon sequestration, that perfectly incorporate gasification into circular economies and sustainable development goals. Despite these features, small scale applications struggle to succeed as a leading solution for sustainable development. The aim of this review is to investigate the existing technological barriers that limit the spreading of biomass gasification from a socio-technical point of view. The review outlines how existing technologies originated from under feed-in-tariff regimes and highlights where the current design goals strongly differ from what will be needed in the near future. Relevant market-ready small-scale gasification systems are analyzed under this lens, leading to an analysis of the reactor and filtration design. To help understand the economical sustainability of these plants, an analysis of the influence of capital expenditures and operating expenditures on the return of investment is included in the discussion. Finally, a literature review on prototypes and pre-market reactors is used as a basis for spotting the characteristics of the system that will likely resolve issues around fuel flexibility, cost efficiency and load variability