Applications of Machine Learning to Optimizing Polyolefin Manufacturing

This chapter is a preprint from our book by , focusing on leveraging machine learning (ML) in chemical and polyolefin manufacturing optimization. It's crafted for both novices and seasoned professionals keen on the latest ML applications in chemical processes. We trace the evolution of AI and ML in chemical industries, delineate core ML components, and provide resources for ML beginners. A detailed discussion on various ML methods is presented, covering regression, classification, and unsupervised learning techniques, with performance metrics and examples. Ensemble methods, deep learning networks, including MLP, DNNs, RNNs, CNNs, and transformers, are explored for their growing role in chemical applications. Practical workshops guide readers through predictive modeling using advanced ML algorithms. The chapter culminates with insights into science-guided ML, advocating for a hybrid approach that enhances model accuracy. The extensive bibliography offers resources for further research and practical implementation. This chapter aims to be a thorough primer on ML's practical application in chemical engineering, particularly for polyolefin production, and sets the stage for continued learning in subsequent chapters. Please cite the original work [169,170] when referencing

Comparison of product quality estimation of propylene polymerization in loop reactors using artificial neural network models

One of the major challenges in polymerization industry is the lack of online instruments to measure polymer end-used properties such as xylene soluble, particle size distribution and melt flow index (MFI). As an alternative to the online instruments and conventional laboratory tests, these properties can be estimated using model based-soft sensor. This paper presents models for soft sensors to measure MFI in industrial polypropylene loop reactors using artificial neural network (ANN) model, serial hybrid neural network (HNN) model and stacked neural network (SNN) model. All models were developed and simulated in MATLAB. The simulation results of the MFI based on the ANN, HNN, and SNN models were compared and analyzed. The MFI was divided into three grades, which are A (10-12g/10 min), B (12-14g/10 min) and C (14-16 g/10 min). The HNN model is the best model in predicting all range of MFI with the lowest root mean square error (RMSE) value, 0.010848, followed by ANN model (RMSE=0.019366) and SNN model (RMSE=0.059132). The SNN model is the best model when tested with each grade of the MFI. It has shown lowest RMSE value for each type of MFI (0.012072 for MFI A, 0.017527 for MFI B and 0.015287 for MFI C), compared to HNN model (0.014916 for MFI A, 0.041402 for MFI B and 0.046437 for MFI C) and ANN model (0.015156 for MFI A, 0.076682 for MFI B, and 0.037862 for MFI C)

Bidirectional optimization of the melting spinning process

This is the author's accepted manuscript (under the provisional title "Bi-directional optimization of the melting spinning process with an immune-enhanced neural network"). The final published article is available from the link below. Copyright 2014 @ IEEE.A bidirectional optimizing approach for the melting spinning process based on an immune-enhanced neural network is proposed. The proposed bidirectional model can not only reveal the internal nonlinear relationship between the process configuration and the quality indices of the fibers as final product, but also provide a tool for engineers to develop new fiber products with expected quality specifications. A neural network is taken as the basis for the bidirectional model, and an immune component is introduced to enlarge the searching scope of the solution field so that the neural network has a larger possibility to find the appropriate and reasonable solution, and the error of prediction can therefore be eliminated. The proposed intelligent model can also help to determine what kind of process configuration should be made in order to produce satisfactory fiber products. To make the proposed model practical to the manufacturing, a software platform is developed. Simulation results show that the proposed model can eliminate the approximation error raised by the neural network-based optimizing model, which is due to the extension of focusing scope by the artificial immune mechanism. Meanwhile, the proposed model with the corresponding software can conduct optimization in two directions, namely, the process optimization and category development, and the corresponding results outperform those with an ordinary neural network-based intelligent model. It is also proved that the proposed model has the potential to act as a valuable tool from which the engineers and decision makers of the spinning process could benefit.National Nature Science Foundation of China, Ministry of Education of China, the Shanghai Committee of Science and Technology), and the Fundamental Research Funds for the Central Universities

Multi-tier framework for the inferential measurement and data-driven modeling

A framework for the inferential measurement and data-driven modeling has been proposed and assessed in several real-world application domains. The architecture of the framework has been structured in multiple tiers to facilitate extensibility and the integration of new components. Each of the proposed four tiers has been assessed in an uncoupled way to verify their suitability. The first tier, dealing with exploratory data analysis, has been assessed with the characterization of the chemical space related to the biodegradation of organic chemicals. This analysis has established relationships between physicochemical variables and biodegradation rates that have been used for model development. At the preprocessing level, a novel method for feature selection based on dissimilarity measures between Self-Organizing maps (SOM) has been developed and assessed. The proposed method selected more features than others published in literature but leads to models with improved predictive power. Single and multiple data imputation techniques based on the SOM have also been used to recover missing data in a Waste Water Treatment Plant benchmark. A new dynamic method to adjust the centers and widths of in Radial basis Function networks has been proposed to predict water quality. The proposed method outperformed other neural networks. The proposed modeling components have also been assessed in the development of prediction and classification models for biodegradation rates in different media. The results obtained proved the suitability of this approach to develop data-driven models when the complex dynamics of the process prevents the formulation of mechanistic models. The use of rule generation algorithms and Bayesian dependency models has been preliminary screened to provide the framework with interpretation capabilities. Preliminary results obtained from the classification of Modes of Toxic Action (MOA) indicate that this could be a promising approach to use MOAs as proxy indicators of human health effects of chemicals.Finally, the complete framework has been applied to three different modeling scenarios. A virtual sensor system, capable of inferring product quality indices from primary process variables has been developed and assessed. The system was integrated with the control system in a real chemical plant outperforming multi-linear correlation models usually adopted by chemical manufacturers. A model to predict carcinogenicity from molecular structure for a set of aromatic compounds has been developed and tested. Results obtained after the application of the SOM-dissimilarity feature selection method yielded better results than models published in the literature. Finally, the framework has been used to facilitate a new approach for environmental modeling and risk management within geographical information systems (GIS). The SOM has been successfully used to characterize exposure scenarios and to provide estimations of missing data through geographic interpolation. The combination of SOM and Gaussian Mixture models facilitated the formulation of a new probabilistic risk assessment approach.Aquesta tesi proposa i avalua en diverses aplicacions reals, un marc general de treball per al desenvolupament de sistemes de mesurament inferencial i de modelat basats en dades. L'arquitectura d'aquest marc de treball s'organitza en diverses capes que faciliten la seva extensibilitat així com la integració de nous components. Cadascun dels quatre nivells en que s'estructura la proposta de marc de treball ha estat avaluat de forma independent per a verificar la seva funcionalitat. El primer que nivell s'ocupa de l'anàlisi exploratòria de dades ha esta avaluat a partir de la caracterització de l'espai químic corresponent a la biodegradació de certs compostos orgànics. Fruit d'aquest anàlisi s'han establert relacions entre diverses variables físico-químiques que han estat emprades posteriorment per al desenvolupament de models de biodegradació. A nivell del preprocés de les dades s'ha desenvolupat i avaluat una nova metodologia per a la selecció de variables basada en l'ús del Mapes Autoorganitzats (SOM). Tot i que el mètode proposat selecciona, en general, un major nombre de variables que altres mètodes proposats a la literatura, els models resultants mostren una millor capacitat predictiva. S'han avaluat també tot un conjunt de tècniques d'imputació de dades basades en el SOM amb un conjunt de dades estàndard corresponent als paràmetres d'operació d'una planta de tractament d'aigües residuals. Es proposa i avalua en un problema de predicció de qualitat en aigua un nou model dinàmic per a ajustar el centre i la dispersió en xarxes de funcions de base radial. El mètode proposat millora els resultats obtinguts amb altres arquitectures neuronals. Els components de modelat proposat s'han aplicat també al desenvolupament de models predictius i de classificació de les velocitats de biodegradació de compostos orgànics en diferents medis. Els resultats obtinguts demostren la viabilitat d'aquesta aproximació per a desenvolupar models basats en dades en aquells casos en els que la complexitat de dinàmica del procés impedeix formular models mecanicistes. S'ha dut a terme un estudi preliminar de l'ús de algorismes de generació de regles i de grafs de dependència bayesiana per a introduir una nova capa que faciliti la interpretació dels models. Els resultats preliminars obtinguts a partir de la classificació dels Modes d'acció Tòxica (MOA) apunten a que l'ús dels MOA com a indicadors intermediaris dels efectes dels compostos químics en la salut és una aproximació factible.Finalment, el marc de treball proposat s'ha aplicat en tres escenaris de modelat diferents. En primer lloc, s'ha desenvolupat i avaluat un sensor virtual capaç d'inferir índexs de qualitat a partir de variables primàries de procés. El sensor resultant ha estat implementat en una planta química real millorant els resultats de les correlacions multilineals emprades habitualment. S'ha desenvolupat i avaluat un model per a predir els efectes carcinògens d'un grup de compostos aromàtics a partir de la seva estructura molecular. Els resultats obtinguts desprès d'aplicar el mètode de selecció de variables basat en el SOM milloren els resultats prèviament publicats. Aquest marc de treball s'ha usat també per a proporcionar una nova aproximació al modelat ambiental i l'anàlisi de risc amb sistemes d'informació geogràfica (GIS). S'ha usat el SOM per a caracteritzar escenaris d'exposició i per a desenvolupar un nou mètode d'interpolació geogràfica. La combinació del SOM amb els models de mescla de gaussianes dona una nova formulació al problema de l'anàlisi de risc des d'un punt de vista probabilístic

Spatial snow water equivalent estimation for mountainous areas using wireless-sensor networks and remote-sensing products

We developed an approach to estimate snow water equivalent (SWE) through interpolation of spatially representative point measurements using a k-nearest neighbors (k-NN) algorithm and historical spatial SWE data. It accurately reproduced measured SWE, using different data sources for training and evaluation. In the central-Sierra American River basin, we used a k-NN algorithm to interpolate data from continuous snow-depth measurements in 10 sensor clusters by fusing them with 14 years of daily 500-m resolution SWE-reconstruction maps. Accurate SWE estimation over the melt season shows the potential for providing daily, near real-time distributed snowmelt estimates. Further south, in the Merced-Tuolumne basins, we evaluated the potential of k-NN approach to improve real-time SWE estimates. Lacking dense ground-measurement networks, we simulated k-NN interpolation of sensor data using selected pixels of a bi-weekly Lidar-derived snow water equivalent product. k-NN extrapolations underestimate the Lidar-derived SWE, with a maximum bias of −10 cm at elevations below 3000 m and +15 cm above 3000 m. This bias was reduced by using a Gaussian-process regression model to spatially distribute residuals. Using as few as 10 scenes of Lidar-derived SWE from 2014 as training data in the k-NN to estimate the 2016 spatial SWE, both RMSEs and MAEs were reduced from around 20–25 cm to 10–15 cm comparing to using SWE reconstructions as training data. We found that the spatial accuracy of the historical data is more important for learning the spatial distribution of SWE than the number of historical scenes available. Blending continuous spatially representative ground-based sensors with a historical library of SWE reconstructions over the same basin can provide real-time spatial SWE maps that accurately represents Lidar-measured snow depth; and the estimates can be improved by using historical Lidar scans instead of SWE reconstructions

Towards a Digital Twin Framework in Additive Manufacturing: Machine Learning and Bayesian Optimization for Time Series Process Optimization

Laser-directed-energy deposition (DED) offers advantages in additive manufacturing (AM) for creating intricate geometries and material grading. Yet, challenges like material inconsistency and part variability remain, mainly due to its layer-wise fabrication. A key issue is heat accumulation during DED, which affects the material microstructure and properties. While closed-loop control methods for heat management are common in DED research, few integrate real-time monitoring, physics-based modeling, and control in a unified framework. Our work presents a digital twin (DT) framework for real-time predictive control of DED process parameters to meet specific design objectives. We develop a surrogate model using Long Short-Term Memory (LSTM)-based machine learning with Bayesian Inference to predict temperatures in DED parts. This model predicts future temperature states in real time. We also introduce Bayesian Optimization (BO) for Time Series Process Optimization (BOTSPO), based on traditional BO but featuring a unique time series process profile generator with reduced dimensions. BOTSPO dynamically optimizes processes, identifying optimal laser power profiles to attain desired mechanical properties. The established process trajectory guides online optimizations, aiming to enhance performance. This paper outlines the digital twin framework's components, promoting its integration into a comprehensive system for AM.Comment: 12 Pages, 10 Figures, 1 Table, NAMRC Conferenc

Automatic Features Extraction From Time Series Of Passive Microwave Images For Snowmelt Detection Using Deep-Learning – A Bidirectional Long-Short Term Memory Autoencoder (Bi-Lstm-Ae) Approach.

The Antarctic surface snowmelt is prone to the polar climate and is common in its coastal regions. With about 90 percent of the planet\u27s glaciers, if all of the Antarctica glaciers melted, sea levels will rise about 58 meters around the planet. The development of an effective automated ice-sheet snowmelt monitoring system is therefore crucial. Microwave remote sensing instruments, on the one hand, are very sensitive to snowmelt and can see day and night through clouds, allowing us to distinguish melting from dry snow and to better understand when, where, and for how long melting has taken place. On the other hand, deep-learning (DL) algorithms, which can learn from linear and non-linear data in a hierarchical way robust representations and discriminative features, have recently become a hotspot in the field of machine learning and have been implemented with success in the geospatial and remote sensing field. This study demonstrates that deep learning, particularly long-short memory autoencoder architecture (LSTM-AE) is capable of fully exploiting archives of passive microwave time series data. In this thesis, An LSTM-AE algorithm was used to reduce and capture essential relationships between attributes stored as brightness temperature within pixel time series and k-means clustering is applied to cluster the leaned representations. The final output map highlights the melt extent in Antarctica

화학 반응기 시스템을 모델링하기 위한 계산 효율적인 전략

학위논문(박사) -- 서울대학교대학원 : 공과대학 화학생물공학부(에너지환경 화학융합기술전공), 2022.2. 이종민.Many researchers in chemical engineering have been using analytical and computational models to predict the behaviors of systems and use these models to process optimization, design and control. However, until now, researchers are still forced to compromise on model fidelity and accuracy. Although high fidelity models can improve the model accuracy, simulating these models is usually time-consuming, making it difficult to perform optimization. In this thesis, computationally efficient strategies about two system are introduced which can maintain model fidelity but effectively reduce the calculation time. Polymer reactor is selected for the first system and we focused on polymer kinetics. A hybrid approach that combines the method of moments and Monte Carlo simulation to predict the molecular weight distribution of low-density polyethylene for a continuous stirred tank reactor system is proposed. A 'Block,' which is repeating reaction group, is introduced for the calculation cost-effective simulation. This model called the 'block Kinetic Monte Carlo' is ~10 to 32 times faster than Neuhaus’s model. The model can be applied to any steady state system and provide a calculation cost reduction effect, where one reaction is much faster than others; for example, the propagation reaction. Furthermore, we perform a case study on the effects of the system temperature and initiator concentration on the MWD and reaction rate ratio. Based on the simulation results of 180 case studies, we determine a quantitative guideline for the appearance of shoulder, which is a function of the rate ratio of reactions to the propagation reaction. Computational fluid dynamics (CFD) based reactor system is selected for the second system. CFD is an essential tool for solving engineering problem that involves fluid dynamics. Especially in chemical engineering, fluid motion usually has extensive effects on system states such as temperature and component concentration. However, due to the critical issue of long computational times for simulating CFD, application of CFD is limited for many real-time problems such as real-time optimization and process control. In this study, we develop the surrogate model of Continuous stirred tank reactor (CSTR) with Van de Vusse reaction using Physics-informed neural network (PINN) which can train the governing equations of system. We propose PINN architecture that can train every governing equation which chemical reactor system follows and can train multi-reference frame system. Also, we investigate that PINN can resolve the problem of neural network that needs lots of training data, are easily overfitted and cannot contain physical meaning. Furthermore, we modify the original PINN suggested by Raissi in order to solve the memory error and divergence problem with two methods: (1) Mini-batch training; (2) Weighted loss function. We also suggest a similarity based sampling strategy where the accuracy can be improved up to 5 times over the random sampling. This work can provide the guideline for developing the high performance surrogate model of chemical process.화학 공학 분야의 많은 연구자들은 분석 및 계산 모델을 이용하여 시스템의 거동을 해석하고 최적화, 설계 및 제어를 수행하고 있다. 하지만 모델의 정확도와 계산시간은 거래되는 관계에 있어 계산시간이 오래걸리는 문제 때문에 모델의 정확도를 타협할 수 밖에 없는 실상이다. 이 학위논문에서는 두 시스템에 대해 모델의 충실도를 유지하면서도 계산시간을 효과적으로 줄일 수 있는 계산 효율적인 전략을 소개한다. 첫 번째 시스템은 고분자 반응기로 고분자의 반응에 중점을 두고 있다. 연속 교반 탱크 반응기에 대한 저밀도 폴리에틸렌의 분자량 분포를 예측하기 위해 모멘트 방법과 몬테 카를로 시뮬레이션 기법을 결합한 하이브리드 접근 방식이 제안되었다. 계산 효율적인 시뮬레이션을 위해 반복되는 반응들을 집합인 ‘블락’이라는 개념이 새로이 도입되었다. ‘블락 키네틱 몬테 카를로’라고 불리는 이 모델은 Neuhaus가 제안한 모델보다 약 10~32배 빠르다. 이 모델은 모든 정상 상태시스템에 적용할 수 있으며, 특정 반응이 다른 반응들보다 훨씬 빠른 경우에 계산 시간 감소효과를 누릴 수 있다. 또한 시스템의 운전 온도 및 개시제의 농도가 분자량 분포에 미치는 영향에 대해 사례 연구를 수행하였다. 180개의 사례 연구 시뮬레이션을 바탕으로 분자량 분포가 숄더를 보이는 조건에 대한 정량적 지침을 제안하였다. 두번째 시스템은 전산유체역학 기반의 반응기 시스템이다. 전산유체역학은 유체의 흐름을 해석함에 있어 필수적인 기법이다. 특히 화학공학반응기에서 유체의 흐름은 내부의 온도나 농도에 큰 영향을 미친다. 그러나 전산유체역학은 계산시간이 오래걸린다는 단점으로 인해 실시간 최적화 및 공정 제어와 같은 응용에 사용이 제한된다. 이 학위논문에서는 시스템의 지배 방정식을 훈련할 수 있는 물리정보신경망(PINN)을 사용하여 Van de Vusse 반응이 포함된 연속 교반 탱크 반응기의 대리 모델을 개발한다. 화학 반응기 시스템이 따르는 모든 종류의 지배 방정식을 훈련할 수 있으며 다중 참조 프레임 시스템을 훈련 할 수 있는 물리정보신경망 모델 구조를 제안한다. 물리정보신경망(PINN)은 기존에 신경망 모델이 가지는 과적합 문제나 데이터가 많이 필요하다는 점 그리고 물리적 의미를 반영할 수 없다는 문제들을 해결할 수 있다. 또한 메모리 오류 및 모델의 발산 문제를 해결하기 위하여 Raissi가 제안한 기존의 물리정보신경망(PINN)을 두 가지 방법으로 수정하였다. 1) 미니 배치 훈련; 2) 가중 손실 함수. 그리고 학습 데이터를 추출함에 있어 무작위 추출에 비해 정확도를 최대 5배까지 향상시킬 수 있는 유사성 기반 추출 전략을 제안한다. 이 연구가 화학 공정의 고성능 대리 모델 개발을 위한 지침이 되기를 희망한다.Abstract i Contents iv List of Figures vii List of Tables xi Chapter 1 Introduction 1 1.1 Research motivation 1 1.2 Research objective 3 1.3 Outline of the thesis 5 Chapter 2 Molecular weight distribution modeling of LDPE in a continuous stirred-tank reactor using coupled deterministic and stochastic approach 6 2.1 Introduction 6 2.2 Methodology 10 2.2.1 Polymer reaction mechanism 10 2.2.2 Reactor model 16 2.2.3 Deterministic part 16 2.2.4 Stochastic part 20 2.3 Result 34 2.3.1 Verification 34 2.3.2 Reduction in calculation time 39 2.3.3 Case study 41 2.3.4 Shouldering condition 49 2.4 Conclusions 52 2.5 Notations 54 2.6 Abbreviations 57 Chapter 3 Physics-informed deep learning for data-driven solutions of computational fluid dynamics 58 3.1 Introduction 58 3.2 PINN 61 3.3 Model description 64 3.3.1 CFD modeling 64 3.3.2 Governing equations 67 3.3.3 PINN architecture 71 3.4 Result and Discussion 79 3.4.1 Model verification 79 3.4.2 Improvement of model performance 86 3.4.3 Comparison of PINN model with 1-D ODE model 98 3.5 Conclusion 102 3.6 Appendix 105 3.7 Notations 106 Chapter 4 Concluding Remarks 111 4.1 Summary of contributions 111 4.2 Future work 112 Reference 114 Abstract in Korean (국문초록) 121박