An Application to Predict NOx Emissions under Transient Conditions of a Diesel Engine

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2021.8. 이현수.최근 딥러닝 기술의 발전은 인공지능의 비약적인 도약을 이끌었다. 딥러닝 기술은 기존의 모델링 방법론과는 다른 접근 방법으로 다양한 분야에서 높은 정확도를 보여주고 있어, 많은 연구자들에게 주목받고 있다. 자동차 및 내연기관 연구 분야에도 이러한 딥러닝을 이용한 연구가 활발히 진행되기 시작하고 있다. 기존의 연구는 엔진의 현상을 단순화하여 주요한 수식으로 모델링하여 대상 현상을 예측하는 방식을 가진다. 이 경우, 현상을 단순화하는 과정에서 오차가 발생하여 예측 정확도에 한계가 있을 수밖에 없었다. 딥러닝은 이러한 단순화 과정 없이, 데이터에 내재된 변수 간의 관계를 학습하여 현상을 예측하기 때문에 오차가 발생할 여지가 적고, 연구자가 파악하기 어려운 관계까지 파악하여 기존보다 정확한 예측 정확도를 달성할 수 있다. 본 연구에서는 기존의 모델링 방법으로는 예측 정확도에 한계가 있었던 과도 상태에서의 디젤 엔진의 질소산화물을 딥러닝을 통해 예측하였다. 하이퍼파라미터 최적화, 알고리즘 비교, 도메인 전이를 위한 데이터 세트 설계를 포함한 딥러닝 모델을 구축하기 위한 전체 프로세스에 대한 연구를 진행하였다. 딥러닝 모델의 구조인 하이퍼파라미터는 베이지안 최적화와 은닉 노드 결정 로직을 결합하여 데이터의 입출력 종류나 개수와 관계없이 자동으로 최적화하였다. 최적화 대상 하이퍼파라미터는 학습률, 학습률의 감쇄율, 배치크기, 은닉 층의 개수, 첫 번째 은닉층의 노드 개수였다. 베이지안 최적화 방법은 최적화 과정에서 이전 정보를 활용하는 베이지안 룰을 기반으로 동작하므로, 기존의 그리드 검색, 랜덤 샘플링에 비해 효과적이고 정확도가 높았다. 은닉 노드 결정 로직은 은닉층의 개수와 첫 번째 은닉층의 노드 개수를 이용하여, 은닉층의 노드 배열을 등차수열로 배치한다. 이를 통해 노드 개수의 급격한 변화로 인한 정보 손실을 막고, 각 은닉층과 노드를 개별적으로 최적화할 경우 기하급수적으로 발산하는 최적화 횟수를 제어할 수 있었다. 과도 상태에서의 질소산화물 예측에 적합한 딥러닝 모델 구조에 관한 연구도 진행되었다. Deep neural network (DNN) 모델과 Long short-term memory (LSTM) 모델의 정확도 및 계산 시간을 실시간 예측 관점에서 평가하였다. LSTM 모델은 DNN 모델에 비해 예측 정확도가 높았으나, 계산에 더 많은 시간이 소요되어 실시간 예측으로의 적용에는 적절하지 않았다. 또한 데이터 전처리를 통해 DNN 모델의 빠른 계산 속도의 장점을 유지하면서, 계산 시간의 정확도를 LSTM 모델과 비슷한 수준으로 높일 수 있었다. 이는 딥러닝을 통해 과도 상태의 질소산화물을 예측할 시, 준 정상 상태를 통한 예측이 가능함을 의미하였다. 마지막으로, 딥러닝을 통해 질소산화물 예측 시, 기존의 학습데이터와 예측데이터의 상태가 동일해야 한다는 제약을 극복하기 위한 연구를 진행하였다. 이를 위해 정상 상태에서의 실험 데이터 세트를 설계하여 모델 학습에 사용함으로써 과도 상태에서의 질소산화물을 예측하였다. 과도 상태의 엔진 거동을 분석하여, 흡기 공기량, 흡기 압력, 분사 압력 및 주 분사 타이밍을 스윙 변수로 설정하여 각 운전점 별로 일정 범위 내에서 실험하여 데이터를 취득하였다. 이를 통해 정상 상태 데이터를 과도 상태 예측으로 확장할 수 있었다. 또한 과도 조건에서 흡기 온도와 냉각수 온도의 영향을 고려하기 위하여 온도 실험을 추가로 설계하고 수행하여, 온도가 다른 조건에서의 과도 상태 질소산화물을 예측하였다. 이러한 과정을 통해 과도 상태 예측을 위한 정상 상태 실험 조건 설계에 대한 방법을 제시하였고, 그 결과 역시 확인할 수 있었다. 이러한 방법으로 딥러닝 모델의 도메인 제한을 극복함으로써, 질소산화물 예측 모델을 엔진 설계, 검증, 예측 등의 개발 단계 전반과 과도 데이터를 측정할 수 있는 실제 엔진이 없는 경우에도 사용할 수 있게 되었다. 또한 본 연구에서 제시된 정상 상태 실험 계획 방법론은 향후 실도로 조건에서의 질소산화물 예측에 적용할 수 있을 것이며, 다른 기계 시스템의 예측에도 활용할 수 있다.Recently, the development of deep learning technology has been leading the rising of artificial intelligence. Deep learning technology is a different approach from conventional modeling methodologies and presents high prediction accuracy in various research fields, which attracted attention from many researchers. In automotive and internal combustion engine research, studies using deep learning are beginning to be actively performed. The conventional methodologies simplified engine phenomena and predicted target phenomena with several equations, which causes errors, and is a limitation of the conventional methodologies. Deep learning predicts phenomena by learning the internal relationship between variables of the data, and it can catch complex relationships that researchers could not recognize. In this study, nitrogen oxides (NOx) under transient conditions in a diesel engine were predicted through deep learning, which had limitations in prediction accuracy using conventional modeling methodologies. This study included the entire process for constructing a deep learning model such as hyperparameter optimization, algorithm comparison, and dataset design for domain transfer. Hyperparameters, the structure of the deep learning model, were automatically optimized by combining the Bayesian optimization and hidden-node determination logic. This optimization process could be conducted regardless of the type or number of data inputs and outputs. The hyperparameters to be optimized were the learning rate, learning rate decay, batch size, number of hidden layers, and number of nodes in 1st hidden layer. Because the Bayesian optimization method utilized the previous information based on the Bayesian rule during the optimization process, it was more effective and more accurate than the conventional grid search and random sampling. The hidden-node determination logic used the number of hidden layers and the number of nodes in 1st hidden layer to arrange the node sequence of hidden layers in an arithmetical sequence. It was possible to prevent information loss due to sudden changes in nodes, and control the number of exponentially increased iterations when each layer and node were individually optimized. A study on the structure of a deep learning model suitable to predict NOx in transient conditions was also conducted. The accuracy and computation time of the deep neural network (DNN) model and long short-term memory (LSTM) model were evaluated from the viewpoint of real-time prediction. The LSTM model presented higher prediction accuracy than that of the DNN model, but it needed more calculation time, which is not suitable for real-time prediction. Through data preprocessing, the accuracy of the DNN model increased to a level similar to that of the LSTM model, and its advantage of the calculation speed is maintained, which is suitable to predict NOx emissions under transient conditions through quasi-stationary. Finally, a study was performed to overcome the domain constraint to predict NOx using deep learning. Previously, the domains of both training data and target data should be identical. In order to predict transient NOx emissions using the model trained by the steady-state data, the experimental dataset under the steady-state conditions was designed and used for model training. From the analysis of the engine behaviors under transient conditions, the intake air mass, intake pressure, injection pressure, and main injection timing were set as swing variables, and data were acquired by experiments in certain ranges for operating points. This allowed the steady-state data to be extended to the transient prediction. Temperature swing experiments were additionally designed and performed to consider the effects of intake and coolant temperature in transient conditions. This process was proposed as a method for designing steady-state experimental conditions for predicting transient conditions, and the results were also provided. By overcoming the domain constraint of deep learning models using this method, a deep learning model for the NOx emission prediction can be used at the overall development stages of engines including design, evaluation, and prediction of whether real engines for measurement of transient NOx emissions exist. In addition, the proposed procedure of the steady-state experimental design for predicting transient NOx emissions could be applied to real driving conditions and other mechanical systems.Chapter 1. Introduction 1 1.1 Background and motivation 1 1.1.1 Status overview of internal combustion engines 1 1.1.2 Issues of the internal combustion engines 2 1.1.3 Rising of deep learning 3 1.2 Literature review 6 1.2.1 Conventional models for NOx emission prediction 6 1.2.2 Deep learning research for prediction of ICE 8 1.2.3 Domain discordance between steady-state and transient conditions 13 1.3 Objectives 16 Chapter 2. Deep learning algorithms 19 Chapter 3. Experimental setup 23 3.1 Hyperparameter optimization and deep learning algorithms 23 3.2 Steady-state experimental design 30 3.3 Computing environment 36 Chapter 4. Hyperparameter optimization using the Bayesian optimization and hidden-node determination logic 37 4.1 Hyperparameter optimization methods 39 4.1.1 General optimization methods 39 4.1.2 Bayesian optimization method 41 4.1.3 Target hyperparameters and the hidden-node determination logic 45 4.2 Optimization model setup 50 4.3 Results of the developed method 53 4.3.1 Optimization process 53 4.3.2 Accuracy analysis 60 Chapter 5. Deep learning algorithms and time-series data preprocessing 68 5.1 Methodology 69 5.1.1 NOx emission prediction sequence of LSTM 69 5.1.2 Data preprocessing 71 5.1.3 Model hyperparameters 73 5.1.4 Dataset assignment 77 5.2 Results 79 5.2.1 Comparison between DNN and LSTM 79 5.2.2 Data-preprocessing results 87 Chapter 6. Steady-state experimental design for prediction of transient NOx emissions 95 6.1 Overview and necessity of steady-state experimental design 95 6.2 DNN model setup 98 6.3 Results and discussion 101 6.3.1 Results of model trained with map experimental data 101 6.3.2 Design of steady-state experimental condition for prediction of transient cycle 115 6.3.3 Design of additional experimental points for considering intake and coolant temperatures 136 6.3.4 Case Study: Application of Dataset Designing Methodology to RDE 146 Chapter 7. Conclusions 151 7.1 Hyperparameter optimization using the Bayesian optimization and hidden-node determination logic 152 7.2 Deep learning algorithms and time-series data preprocessing 153 7.3 Steady-state experimental design for prediction of transient NOx emissions 154 국 문 초 록 166박

Prediction of indicators for solid waste management on national level using artificial neural networks.

Problem upravljanja otpadom postoji od nastanka najranijih ljudskih naseobina, ali je posebno izražen u današnjim urbanim sredinama. Sa povećanjem gustine naseljenosti do koje dolazi usled porasta broja stanovnika i njihovog gravitiranja ka velikim gradovima, kao i sa ubrzanim napredovanjem industrije, savremeni čovek stvara daleko više otpada nego ikada u istoriji ljudskog društva. Napredak nauke i tehnologije dovodi do stvaranja novih, najraznovrsnijih proizvoda, koji, usled porasta životnog standarda i izmenjenih potrošačkih navika, imaju znatno kraći životni vek nego raniji proizvodi, samim tim i mnogo ranije postaju otpad. Sve ovo značajno usložnjava sakupljanje i tretman otpada. Sa druge strane, kao nikad do sada, otpad predstavlja značajan resurs koji može da se iskoristi za dobijanje energije ili novih proizvoda raznovrsne namene...Dealing with waste is a problem since the emergence of the earliest human settlements, but it is particularly pronounced in contemporary urban areas. With the increase in population density due to population growth and their gravitation towards big cities, as well as with the rapid advancement of industry, modern man creates far more waste than ever in the history of human society. Big progress of science and technology leads to the creation of new, more diversified products, which, due to the rise in living standards and altered consumer habits, have a much shorter life expectancy than earlier products, thus becoming much earlier a waste. All mentioned significantly complicates the collection and treatment of waste. On the other hand, as never before, waste represents a significant resource that can be used to generate energy or new products..

An informatics based approach to respiratory healthcare.

By 2005 one person in every five UK households suffered with asthma. Research has shown that episodes of poor air quality can have a negative effect on respiratory health and is a growing concern for the asthmatic. To better inform clinical staff and patients to the contribution of poor air quality on patient health, this thesis defines an IT architecture that can be used by systems to identify environmental predictors leading to a decline in respiratory health of an individual patient. Personal environmental predictors of asthma exacerbation are identified by validating the delay between environmental predictors and decline in respiratory health. The concept is demonstrated using prototype software, and indicates that the analytical methods provide a mechanism to produce an early warning of impending asthma exacerbation due to poor air quality. The author has introduced the term enviromedics to describe this new field of research. Pattern recognition techniques are used to analyse patient-specific environments, and extract meaningful health predictors from the large quantities of data involved (often in the region of '/o million data points). This research proposes a suitable architecture that defines processes and techniques that enable the validation of patient-specific environmental predictors of respiratory decline. The design of the architecture was validated by implementing prototype applications that demonstrate, through hospital admissions data and personal lung function monitoring, that air quality can be used as a predictor of patient-specific health. The refined techniques developed during the research (such as Feature Detection Analysis) were also validated by the application prototypes. This thesis makes several contributions to knowledge, including: the process architecture; Feature Detection Analysis (FDA) that automates the detection of trend reversals within time series data; validation of the delay characteristic using a Self-organising Map (SOM) that is used as an unsupervised method of pattern recognition; Frequency, Boundary and Cluster Analysis (FBCA), an additional technique developed by this research to refine the SOM

Electronic Noses for Biomedical Applications and Environmental Monitoring

This book, titled “Electronic Noses for Biomedical Applications and Environmental Monitoring”, includes original research works and reviews concerning the use of electronic nose technology in two of the more useful and interesting fields related to chemical compounds detection of gases. Authors have explained their latest research work, including different gas sensors and materials based on nanotechnology and novel applications of electronic noses for the detection of diverse diseases. Some reviews related to disease detection through breath analysis, odor monitoring systems standardization, and seawater quality monitoring are also included

Lernbeiträge im Rahmen einer kognitiven Architektur für die intelligente Prozessführung

In dieser Arbeit werden wichtige Aspekte einer kognitiven Architektur für das Erlernen von Regelungsaufgaben beleuchtet. Dabei geht es primär um die Merkmalsextraktion, das Reinforcement Learning und das Lernmanagement im Rahmen des Wahrnehmungs-Handlungs-Zyklus. Wichtige Beiträge sind dabei verschiedene residuumsbasierte Ansätze zur hybriden Merkmalsselektion, ein Algorithmus zur Behandlung des Explorations-Exploitation-Dilemmas in kontinuierlichen Aktionsräumen, Untersuchungen zum Rewarddekompositionsproblem, sowie die Verzahnung der einzelnen Komponenten einer funktionierenden Architektur. Der experimentelle Nachweis, dass das vorgestellte System die Lösung für reale Probleme erlernen kann, wird am herausfordernden Szenario der intelligenten Feuerungsführung erbracht. Dabei wird das Gesamtsystem zur Regelung eines mit Steinkohle gefeuerten Kraftwerks eingesetzt. Dabei wurden Ergebnisse erzielt, die bisher existierende Systeme und auch menschliche Experten deutlich übertreffen.In this thesis, important aspects of a cognitive architecture for learning control tasks are discussed. Highlighted are the topics of feature extraction, reinforcement learning and learning management in the context of the perception-action-cycle. The contributions in the field of feature extraction utilize informationtheoretic measures such as mutual information to formulate new hybrid feature extraction algorithms. Finding features that are explicitly linked with the errors made by a learning system are the focus. It is shown this approach based on residuals is superior to classical methods. Another topic of interest is the estimation of mutual information in the context of feature extraction. State of the art reinforcement learning methods are investigated for their suitability for challenging applications. This work addresses issues of learning management, such as the exploration-exploitation dilemma, the plasticity-stability dilemma and the reward decomposition problem. New contributions are made in the form of the diffusion tree-based reinforcement learning algorithm and the SMILE approach. Likewise, an architectural extension is proposed to organize the learning process. It uses a process map as the core piece to achieve this organization. Experimental evidence that the proposed system can learn the solution to real problems is presented in the challenging scenario of intelligent combustion control. The system is used to learn a control strategy in a coal-fired power plant. The achieved results surpass existing systems and human experts.In dieser Arbeit werden wichtige Aspekte einer kognitiven Architektur für das Erlernen von Regelungsaufgaben beleuchtet. Dabei geht es primär um die Merkmalsextraktion, das Reinforcement Learning und das Lernmanagement im Rahmen des Wahrnehmungs-Handlungs-Zyklus. Für die Merkmalsextraktion werden dabei mit Hilfe informationstheoretischer Größen, wie der Transinformation, neue hybride Merkmalsextraktionsverfahren vorgestellt. Neuartig ist dabei der Ansatz, Merkmale zu suchen, die explizit mit den gemachten Fehlern eines lernenden Systems verknüpft sind. Es wird gezeigt, dass diese residuumsbasierten Ansätze klassischen Methoden überlegen sind. Es wird ebenfalls untersucht, welche Schätzverfahren für die Bestimmung der Transinformation im Sinne der Merkmalsextraktion geeignet sind. Als Entscheidungsinstanz der Gesamtarchitektur werden aktuelle Reinforcement Learning Verfahren auf ihre Eignung für komplexe Anwendungen hin untersucht. Dabei wird auch auf Probleme des Lernmanagements, wie das Explorations-Exploitations-Dilemma, das Stabilitäts-Plastizitäts-Dilemma und das Rewarddekompositionsproblem eingegangen. Neue Beiträge werden dabei in Form des Diffusionsbaumbasiertes Reinforcement Learning und des SMILE-Algorithmus geliefert. Ebenso wird eine Architekturerweiterung zum Organisieren der Lernprozesse vorgeschlagen, welche im Kern um eine Prozesskarte angeordnet ist. Der experimentelle Nachweis, dass das vorgestellte System die Lösung für reale Probleme erlernen kann, wird am herausfordernden Szenarioder intelligenten Feuerungsführung erbracht. Dabei wird das Gesamtsystem zur Regelung eines mit Steinkohle gefeuerten Kraftwerks eingesetzt, wobei Ergebnisse erzielt werden, die bisher existierende Systeme und auch menschliche Experten übertreffen

1985 April, Memphis State University bulletin

Vol. 74, No. 1 of the Memphis State University bulletin containing the undergraduate catalog for 1985-86, 1985 April.https://digitalcommons.memphis.edu/speccoll-ua-pub-bulletins/1160/thumbnail.jp

A hybrid electronic nose system for monitoring the quality of potable water

This PhD thesis reports on the potential application of an electronic nose to analysing the quality of potable water. The enrichment of water by toxic cyanobacteria is fast becoming a severe problem in the quality of water and a common source of environmental odour pollution. Thus, of particular interest is the classification and early warning of toxic cyanobacteria in water. This research reports upon the first attempt to identify electronically cyanobacteria in water. The measurement system comprises a Cellfacts instrument and a Warwick e-nose specially constructed for the testing of the cyanobacteria in water. The Warwick e- nose employed an array of six commercial odour sensors and was set-up to monitor not only the different strains, but also the growth phases, of cyanobacteria. A series of experiments was carried out to analyse the nature of two closely related strains of cyanobacteria, Microcystis aeruginosa PCC 7806 which produces a toxin and PCC 7941 that does not. Several pre-processing techniques were explored in order to remove the noise factor associated with running the electronic nose in ambient air, and the normalised fractional difference method was found to give the best PCA plot. Three supervised neural networks, MLP, LVQ and Fuzzy ARTMAP, were used and compared for the classification of both two strains and four different growth phases of cyanobacteria (lag, growth, stationary and late stationary). The optimal MLP network was found to classify correctly 97.1 % of unknown non-toxic and 100 % of unknown toxic cyanobacteria. The optimal LVQ and Fuzzy ARTMAP algorithms were able to classify 100% of both strains of cyanobacteria. The accuracy of MLP, LVQ and Fuzzy ARTMAP algorithms with 4 different growth phases of toxic cyanobacteria was 92.3 %, 95.1 % and 92.3 %, respectively. A hybrid e-nose system based on 6 MOS, 6 CP, 2 temperature sensors, 1 humidity sensor and 2 flow sensors was finally developed. Using the hybrid system, data were gathered on six different cyanobacteria cultures for the classification of growth phase. The hybrid resistive nose showed high resolving power to discriminate six growth stages as well as three growth phases. Even though time did not permit many series of the continuous monitoring, because of the relatively long life span (30-40 days) of cyanobacteria, improved results indicate the use of a hybrid nose. The HP 4440 chemical sensor was also used for the discrimination of six different cyanobacteria samples and the comparison with the electronic nose. The hybrid resistive nose based on 6 MOS and 6 CP showed a better resolving power to discriminate six growth stages as well as three growth phases than the HP 4440 chemical sensor. Although the mass analyser detects individual volatile chemicals accurately, it proves no indication of whether the volatile is an odour. The results demonstrate that it is possible to apply the e-nose system for monitoring the quality of potable water. It would be expected that the hybrid e-nose could be applicable to a large number of applications in health and safety with a greater flexibility

1982 June, Memphis State University bulletin

Vol. 71, No. 1 of the Memphis State University bulletin containing the undergraduate catalog for 1982-83, 1982 June.https://digitalcommons.memphis.edu/speccoll-ua-pub-bulletins/1155/thumbnail.jp

An informatics based approach to respiratory healthcare

By 2005 one person in every five UK households suffered with asthma. Research has shown that episodes of poor air quality can have a negative effect on respiratory health and is a growing concern for the asthmatic. To better inform clinical staff and patients to the contribution of poor air quality on patient health, this thesis defines an IT architecture that can be used by systems to identify environmental predictors leading to a decline in respiratory health of an individual patient. Personal environmental predictors of asthma exacerbation are identified by validating the delay between environmental predictors and decline in respiratory health. The concept is demonstrated using prototype software, and indicates that the analytical methods provide a mechanism to produce an early warning of impending asthma exacerbation due to poor air quality. The author has introduced the term enviromedics to describe this new field of research. Pattern recognition techniques are used to analyse patient-specific environments, and extract meaningful health predictors from the large quantities of data involved (often in the region of '/o million data points). This research proposes a suitable architecture that defines processes and techniques that enable the validation of patient-specific environmental predictors of respiratory decline. The design of the architecture was validated by implementing prototype applications that demonstrate, through hospital admissions data and personal lung function monitoring, that air quality can be used as a predictor of patient-specific health. The refined techniques developed during the research (such as Feature Detection Analysis) were also validated by the application prototypes. This thesis makes several contributions to knowledge, including: the process architecture; Feature Detection Analysis (FDA) that automates the detection of trend reversals within time series data; validation of the delay characteristic using a Self-organising Map (SOM) that is used as an unsupervised method of pattern recognition; Frequency, Boundary and Cluster Analysis (FBCA), an additional technique developed by this research to refine the SOM.EThOS - Electronic Theses Online ServiceGBUnited Kingdo