Oil and Gas flow Anomaly Detection on offshore naturally flowing wells using Deep Neural Networks

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data ScienceThe Oil and Gas industry, as never before, faces multiple challenges. It is being impugned for being dirty, a pollutant, and hence the more demand for green alternatives. Nevertheless, the world still has to rely heavily on hydrocarbons, since it is the most traditional and stable source of energy, as opposed to extensively promoted hydro, solar or wind power. Major operators are challenged to produce the oil more efficiently, to counteract the newly arising energy sources, with less of a climate footprint, more scrutinized expenditure, thus facing high skepticism regarding its future. It has to become greener, and hence to act in a manner not required previously. While most of the tools used by the Hydrocarbon E&P industry is expensive and has been used for many years, it is paramount for the industry’s survival and prosperity to apply predictive maintenance technologies, that would foresee potential failures, making production safer, lowering downtime, increasing productivity and diminishing maintenance costs. Many efforts were applied in order to define the most accurate and effective predictive methods, however data scarcity affects the speed and capacity for further experimentations. Whilst it would be highly beneficial for the industry to invest in Artificial Intelligence, this research aims at exploring, in depth, the subject of Anomaly Detection, using the open public data from Petrobras, that was developed by experts. For this research the Deep Learning Neural Networks, such as Recurrent Neural Networks with LSTM and GRU backbones, were implemented for multi-class classification of undesirable events on naturally flowing wells. Further, several hyperparameter optimization tools were explored, mainly focusing on Genetic Algorithms as being the most advanced methods for such kind of tasks. The research concluded with the best performing algorithm with 2 stacked GRU and the following vector of hyperparameters weights: [1, 47, 40, 14], which stand for timestep 1, number of hidden units 47, number of epochs 40 and batch size 14, producing F1 equal to 0.97%. As the world faces many issues, one of which is the detrimental effect of heavy industries to the environment and as result adverse global climate change, this project is an attempt to contribute to the field of applying Artificial Intelligence in the Oil and Gas industry, with the intention to make it more efficient, transparent and sustainable

Developing dynamic machine learning surrogate models of physics-based industrial process simulation models

Abstract. Dynamic physics-based models of industrial processes can be computationally heavy which prevents using them in some applications, e.g. in process operator training. Suitability of machine learning in creating surrogate models of a physics-based unit operation models was studied in this research. The main motivation for this was to find out if machine learning model can be accurate enough to replace the corresponding physics-based components in dynamic modelling and simulation software Apros® which is developed by VTT Technical Research Centre of Finland Ltd and Fortum. This study is part of COCOP project, which receive funding from EU, and INTENS project that is Business Finland funded. The research work was divided into a literature study and an experimental part. In the literature study, the steps of modelling with data-driven methods were studied and artificial neural network architectures suitable for dynamic modelling were investigated. Based on that, four neural network architectures were chosen for the case studies. In the first case study, linear and nonlinear autoregressive models with exogenous inputs (ARX and NARX respectively) were used in modelling dynamic behaviour of a water tank process build in Apros®. In the second case study, also Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) were considered and compared with the previously mentioned ARX and NARX models. The workflow from selecting the input and output variables for the machine learning model and generating the datasets in Apros® to implement the machine learning models back to Apros® was defined. Keras is an open source neural network library running on Python that was utilised in the model generation framework which was developed as a part of this study. Keras library is a very popular library that allow fast experimenting. The framework make use of random hyperparameter search and each model is tested on a validation dataset in dynamic manner, i.e. in multi-step-ahead configuration, during the optimisation. The best models based in terms of average normalised root mean squared error (NRMSE) is selected for further testing. The results of the case studies show that accurate multi-step-ahead models can be built using recurrent artificial neural networks. In the first case study, the linear ARX model achieved slightly better NRMSE value than the nonlinear one, but the accuracy of both models was on a very good level with the average NRMSE being lower than 0.1 %. The generalisation ability of the models was tested using multiple datasets and the models proved to generalise well. In the second case study, there were more difference between the models’ accuracies. This was an expected result as the studied process contains nonlinearities and thus the linear ARX model performed worse in predicting some output variables than the nonlinear ones. On the other hand, ARX model performed better with some other output variables. However, also in the second case study the model NRMSE values were on good level, being 1.94–3.60 % on testing dataset. Although the workflow to implement machine learning models in Apros® using its Python binding was defined, the actual implementation need more work. Experimenting with Keras neural network models in Apros® was noticed to slow down the simulation even though the model was fast when testing it outside of Apros®. The Python binding in Apros® do not seem to cause overhead to the calculation process which is why further investigating is needed. It is obvious that the machine learning model must be very accurate if it is to be implemented in Apros® because it needs to be able interact with the physics-based model. The actual accuracy requirement that Apros® sets should be also studied to know if and in which direction the framework made for this study needs to be developed.Dynaamisten surrogaattimallien kehittäminen koneoppimismenetelmillä teollisuusprosessien fysiikkapohjaisista simulaatiomalleista. Tiivistelmä. Teollisuusprosessien toimintaa jäljittelevät dynaamiset fysiikkapohjaiset simulaatiomallit voivat laajuudesta tai yksityiskohtien määrästä johtuen olla laskennallisesti raskaita. Tämä voi rajoittaa simulaatiomallin käyttöä esimerkiksi prosessioperaattorien koulutuksessa ja hidastaa simulaattorin avulla tehtävää prosessien optimointia. Tässä tutkimuksessa selvitettiin koneoppimismenetelmillä luotujen mallien soveltuvuutta fysiikkapohjaisten yksikköoperaatiomallien surrogaattimallinnukseen. Fysiikkapohjaiset mallit on luotu teollisuusprosessien dynaamiseen mallinnukseen ja simulointiin kehitetyllä Apros®-ohjelmistolla, jota kehittää Teknologian tutkimuskeskus VTT Oy ja Fortum. Työ on osa COCOP-projektia, joka saa rahoitusta EU:lta, ja INTENS-projektia, jota rahoittaa Business Finland. Työ on jaettu kirjallisuusselvitykseen ja kahteen kokeelliseen case-tutkimukseen. Kirjallisuusosiossa selvitettiin datapohjaisen mallinnuksen eri vaiheet ja tutkittiin dynaamiseen mallinnukseen soveltuvia neuroverkkorakenteita. Tämän perusteella valittiin neljä neuroverkkoarkkitehtuuria case-tutkimuksiin. Ensimmäisessä case-tutkimuksessa selvitettiin lineaarisen ja epälineaarisen autoregressive model with exogenous inputs (ARX ja NARX) -mallin soveltuvuutta pinnankorkeuden säädöllä varustetun vesisäiliömallin dynaamisen käyttäytymisen mallintamiseen. Toisessa case-tutkimuksessa tarkasteltiin edellä mainittujen mallityyppien lisäksi Long Short-Term Memory (LSTM) ja Gated Recurrent Unit (GRU) -verkkojen soveltuvuutta power-to-gas prosessin metanointireaktorin dynaamiseen mallinnukseen. Työssä selvitettiin surrogaattimallinnuksen vaiheet korvattavien yksikköoperaatiomallien ja siihen liittyvien muuttujien valinnasta datan generointiin ja koneoppimismallien implementointiin Aprosiin. Koneoppimismallien rakentamiseen tehtiin osana työtä Python-sovellus, joka hyödyntää Keras Python-kirjastoa neuroverkkomallien rakennuksessa. Keras on suosittu kirjasto, joka mahdollistaa nopean neuroverkkomallien kehitysprosessin. Työssä tehty sovellus hyödyntää neuroverkkomallien hyperparametrien optimoinnissa satunnaista hakua. Jokaisen optimoinnin aikana luodun mallin tarkkuutta dynaamisessa simuloinnissa mitataan erillistä aineistoa käyttäen. Jokaisen mallityypin paras malli valitaan NRMSE-arvon perusteella seuraaviin testeihin. Case-tutkimuksen tuloksien perusteella neuroverkoilla voidaan saavuttaa korkea tarkkuus dynaamisessa simuloinnissa. Ensimmäisessä case-tutkimuksessa lineaarinen ARX-malli oli hieman epälineaarista tarkempi, mutta molempien mallityyppien tarkkuus oli hyvä (NRMSE alle 0.1 %). Mallien yleistyskykyä mitattiin simuloimalla usealla aineistolla, joiden perusteella yleistyskyky oli hyvällä tasolla. Toisessa case-tutkimuksessa vastemuuttujien tarkkuuden välillä oli eroja lineaarisen ja epälineaaristen mallityyppien välillä. Tämä oli odotettu tulos, sillä joidenkin mallinnettujen vastemuuttujien käyttäytyminen on epälineaarista ja näin ollen lineaarinen ARX-malli suoriutui niiden mallintamisesta epälineaarisia malleja huonommin. Toisaalta lineaarinen ARX-malli oli tarkempi joidenkin vastemuuttujien mallinnuksessa. Kaiken kaikkiaan mallinnus onnistui hyvin myös toisessa case-tutkimuksessa, koska käytetyillä mallityypeillä saavutettiin 1.94–3.60 % NRMSE-arvo testidatalla simuloitaessa. Koneoppimismallit saatiin sisällytettyä Apros-malliin käyttäen Python-ominaisuutta, mutta prosessi vaatii lisäselvitystä, jotta mallit saadaan toimimaan yhdessä. Testien perusteella Keras-neuroverkkojen käyttäminen näytti hidastavan simulaatiota, vaikka neuroverkkomalli oli nopea Aprosin ulkopuolella. Aprosin Python-ominaisuus ei myöskään näytä itsessään aiheuttavan hitautta, jonka takia asiaa tulisi selvittää mallien implementoinnin mahdollistamiseksi. Koneoppimismallin tulee olla hyvin tarkka toimiakseen vuorovaikutuksessa fysiikkapohjaisen mallin kanssa. Jatkotutkimuksen ja Python-sovelluksen kehittämisen kannalta on tärkeää selvittää mikä on Aprosin koneoppimismalleille asettama tarkkuusvaatimus

Modeling and identification of power electronic converters

Nowadays, many industries are moving towards more electrical systems and components. This is done with the purpose of enhancing the efficiency of their systems while being environmentally friendlier and sustainable. Therefore, the development of power electronic systems is one of the most important points of this transition. Many manufacturers have improved their equipment and processes in order to satisfy the new necessities of the industries (aircraft, automotive, aerospace, telecommunication, etc.). For the particular case of the More Electric Aircraft (MEA), there are several power converters, inverters and filters that are usually acquired from different manufacturers. These are switched mode power converters that feed multiple loads, being a critical element in the transmission systems. In some cases, these manufacturers do not provide the sufficient information regarding the functionality of the devices such as DC/DC power converters, rectifiers, inverters or filters. Consequently, there is the need to model and identify the performance of these components to allow the aforementioned industries to develop models for the design stage, for predictive maintenance, for detecting possible failures modes, and to have a better control over the electrical system. Thus, the main objective of this thesis is to develop models that are able to describe the behavior of power electronic converters, whose parameters and/or topology are unknown. The algorithms must be replicable and they should work in other types of converters that are used in the power electronics field. The thesis is divided in two main cores, which are the parameter identification for white-box models and the black-box modeling of power electronics devices. The proposed approaches are based on optimization algorithms and deep learning techniques that use non-intrusive measurements to obtain a set of parameters or generate a model, respectively. In both cases, the algorithms are trained and tested using real data gathered from converters used in aircrafts and electric vehicles. This thesis also presents how the proposed methodologies can be applied to more complex power systems and for prognostics tasks. Concluding, this thesis aims to provide algorithms that allow industries to obtain realistic and accurate models of the components that they are using in their electrical systems.En la actualidad, el uso de sistemas y componentes eléctricos complejos se extiende a múltiples sectores industriales. Esto se hace con el propósito de mejorar su eficiencia y, en consecuencia, ser más sostenibles y amigables con el medio ambiente. Por tanto, el desarrollo de sistemas electrónicos de potencia es uno de los puntos más importantes de esta transición. Muchos fabricantes han mejorado sus equipos y procesos para satisfacer las nuevas necesidades de las industrias (aeronáutica, automotriz, aeroespacial, telecomunicaciones, etc.). Para el caso particular de los aviones más eléctricos (MEA, por sus siglas en inglés), existen varios convertidores de potencia, inversores y filtros que suelen adquirirse a diferentes fabricantes. Se trata de convertidores de potencia de modo conmutado que alimentan múltiples cargas, siendo un elemento crítico en los sistemas de transmisión. En algunos casos, estos fabricantes no proporcionan la información suficiente sobre la funcionalidad de los dispositivos como convertidores de potencia DC-DC, rectificadores, inversores o filtros. En consecuencia, existe la necesidad de modelar e identificar el desempeño de estos componentes para permitir que las industrias mencionadas desarrollan modelos para la etapa de diseño, para el mantenimiento predictivo, para la detección de posibles modos de fallas y para tener un mejor control del sistema eléctrico. Así, el principal objetivo de esta tesis es desarrollar modelos que sean capaces de describir el comportamiento de un convertidor de potencia, cuyos parámetros y/o topología se desconocen. Los algoritmos deben ser replicables y deben funcionar en otro tipo de convertidores que se utilizan en el campo de la electrónica de potencia. La tesis se divide en dos núcleos principales, que son la identificación de parámetros de los convertidores y el modelado de caja negra (black-box) de dispositivos electrónicos de potencia. Los enfoques propuestos se basan en algoritmos de optimización y técnicas de aprendizaje profundo que utilizan mediciones no intrusivas de las tensiones y corrientes de los convertidores para obtener un conjunto de parámetros o generar un modelo, respectivamente. En ambos casos, los algoritmos se entrenan y prueban utilizando datos reales recopilados de convertidores utilizados en aviones y vehículos eléctricos. Esta tesis también presenta cómo las metodologías propuestas se pueden aplicar a sistemas eléctricos más complejos y para tareas de diagnóstico. En conclusión, esta tesis tiene como objetivo proporcionar algoritmos que permitan a las industrias obtener modelos realistas y precisos de los componentes que están utilizando en sus sistemas eléctricos.Postprint (published version

Deep Learning of Microstructures

The internal structure of materials also called the microstructure plays a critical role in the properties and performance of materials. The chemical element composition is one of the most critical factors in changing the structure of materials. However, the chemical composition alone is not the determining factor, and a change in the production process can also significantly alter the materials\u27 structure. Therefore, many efforts have been made to discover and improve production methods to optimize the functional properties of materials. The most critical challenge in finding materials with enhanced properties is to understand and define the salient features of the structure of materials that have the most significant impact on the desired property. In other words, by process, structure, and property (PSP) linkages, the effect of changing process variables on material structure and, consequently, the property can be examined and used as a powerful tool in material design with desirable characteristics. In particular, forward PSP linkages construction has received considerable attention thanks to the sophisticated physics-based models. Recently, machine learning (ML), and data science have also been used as powerful tools to find PSP linkages in materials science. One key advantage of the ML-based models is their ability to construct both forward and inverse PSP linkages. Early ML models in materials science were primarily focused on process-property linkages construction. Recently, more microstructures are included in the materials design ML models. However, the inverse design of microstructures, i.e., the prediction of vii process and chemistry from a microstructure morphology image have received limited attention. This is a critical knowledge gap to address specifically for the problems that the ideal microstructure or morphology with the specific chemistry associated with the morphological domains are known, but the chemistry and processing which would lead to that ideal morphology are unknown. In this study, first, we propose a framework based on a deep learning approach that enables us to predict the chemistry and processing history just by reading the morphological distribution of one element. As a case study, we used a dataset from spinodal decomposition simulation of Fe-Cr-Co alloy created by the phase-field method. The mixed dataset, which includes both images, i.e., the morphology of Fe distribution, and continuous data, i.e., the Fe minimum and maximum concentration in the microstructures, are used as input data, and the spinodal temperature and initial chemical composition are utilized as the output data to train the proposed deep neural network. The proposed convolutional layers were compared with pretrained EfficientNet convolutional layers as transfer learning in microstructure feature extraction. The results show that the trained shallow network is effective for chemistry prediction. However, accurate prediction of processing temperature requires more complex feature extraction from the morphology of the microstructure. We benchmarked the model predictive accuracy for real alloy systems with a Fe-Cr-Co transmission electron microscopy micrograph. The predicted chemistry and heat treatment temperature were in good agreement with the ground truth. The treatment time was considered to be constant in the first study. In the second work, we propose a fused-data deep learning framework that can predict the heat treatment time as well as temperature and initial chemical compositions by reading the morphology of Fe distribution and its concentration. The results show that the trained deep neural network has the highest accuracy for chemistry and then time and temperature. We identified two scenarios for inaccurate predictions; 1) There are several paths for an identical microstructure, and 2) Microstructures reach steady-state morphologies after a long time of aging. The error analysis shows that most of the wrong predictions are not wrong, but the other right answers. We validated the model successfully with an experimental Fe-Cr-Co transmission electron microscopy micrograph. Finally, since the data generation by simulation is computationally expensive, we propose a quick and accurate Predictive Recurrent Neural Network (PredRNN) model for the microstructure evolution prediction. Essentially, microstructure evolution prediction is a spatiotemporal sequence prediction problem, where the prediction of material microstructure is difficult due to different process histories and chemistry. As a case study, we used a dataset from spinodal decomposition simulation of Fe-Cr-Co alloy created by the phase-field method for training and predicting future microstructures by previous observations. The results show that the trained network is capable of efficient prediction of microstructure evolution

Machine learning in bioprocess development: From promise to practice

Fostered by novel analytical techniques, digitalization and automation, modern bioprocess development provides high amounts of heterogeneous experimental data, containing valuable process information. In this context, data-driven methods like machine learning (ML) approaches have a high potential to rationally explore large design spaces while exploiting experimental facilities most efficiently. The aim of this review is to demonstrate how ML methods have been applied so far in bioprocess development, especially in strain engineering and selection, bioprocess optimization, scale-up, monitoring and control of bioprocesses. For each topic, we will highlight successful application cases, current challenges and point out domains that can potentially benefit from technology transfer and further progress in the field of ML

Neural Networks for Modeling and Control of Particle Accelerators

We describe some of the challenges of particle accelerator control, highlight recent advances in neural network techniques, discuss some promising avenues for incorporating neural networks into particle accelerator control systems, and describe a neural network-based control system that is being developed for resonance control of an RF electron gun at the Fermilab Accelerator Science and Technology (FAST) facility, including initial experimental results from a benchmark controller.Comment: 21 p