Machine Learning-based Generalized Multiscale Finite Element Method and its Application in Reservoir Simulation

In multiscale modeling of subsurface fluid flow in heterogeneous porous media, standard polynomial basis functions are replaced by multiscale basis functions, which are used to predict pressure distribution. To produce such functions in the mixed Generalized Multiscale Finite Element Method (GMsFEM), a number of Partial Differential Equations (PDEs) must be solved, leading to significant computational overhead. The main objective of the work presented in this thesis was to investigate the efficiency of Machine Learning (ML)/Deep Learning (DL) models in reconstructing the multiscale basis functions (Basis 2, 3, 4, and 5) of the mixed GMsFEM. To achieve this, four standard models named SkiplessCNN models were first developed to predict four different multiscale basis functions. These predictions were based on two distinct datasets (initial and extended) generated, with the permeability field being the sole input. Subsequently, focusing on the extended dataset, three distinct skip connection schemes (FirstSkip, MidSkip, and DualSkip) were incorporated into the SkiplessCNN architecture. Following this, the four developed models - SkiplessCNN, FirstSkipCNN, MidSkipCNN, and DualSkipCNN - were separately combined using linear regression and ridge regression within the framework of Deep Ensemble Learning (DEL). Furthermore, the reliability of the DualSkipCNN model was examined using Monte Carlo (MC) dropout. Ultimately, two Fourier Neural Operator (FNO) models, operating on infinite-dimensional spaces, were developed based on a new dataset for directly predicting pressure distribution. Based on the results, sufficient data for the validation and testing subsets could help decrease overfitting. Additionally, all three skip connections were found to be effective in enhancing the performance of SkiplessCNN, with DualSkip being the most effective among them. As evaluated on the testing subset, the combined models using linear regression and ridge regression significantly outperformed the individual models for all basis functions. The results also confirmed the robustness of MC dropout for DualSkipCNN in terms of epistemic uncertainty. Regarding the FNO models, it was discovered that the inclusion of a MultiLayer Perceptron (MLP) in the original Fourier layers significantly improved the prediction performance on the testing subset. Looking at this work as an image (matrix)-to-image (matrix) problem, the developed data-driven models through various techniques could find applications beyond reservoir engineering

Reliable predictions of oil formation volume factor based on transparent and auditable machine learning approaches

 Neural-network, machine-learning algorithms are effective prediction tools but can behave as black boxes in many applications by not easily providing the exact calculations and relationships among the underlying input variables (which may or may not be independent of each other) involved each of their predictions. The transparent open box (TOB) learning network algorithm overcomes this limitation by providing the exact calculations involved in all its predictions and achieving acceptable and auditable levels of prediction accuracy. The TOB network, based on an optimized data-matching algorithm, can be applied in spreadsheet or fully-coded configurations. This algorithm offers significant benefits to analysis and prediction of many complex and difficult to measure non-linear systems. To demonstrate its prediction performance, the algorithm is applied to the prediction of crude oil formation volume factor at bubble point (Bob) using published datasets of 166, 203 and 237 data records involving 4 variables (reservoir temperature, gas-oil ratio, oil gravity and gas specific gravity). Two of these datasets display uneven and irregular data coverage. The TOB network demonstrates high prediction accuracy for Bob (Root Mean Square Error (RMSE) ~ 0.03; R2 > 0.95) for the more evenly distributed dataset. The performance of the TOB readily reveals the risk of overfitting such datasets. With its high levels of transparency and inhibitions to being overfitted, the TOB learning network offers an insightful approach to machine learning applied to predicting complex non-linear systems. Its results complement and benchmark the prediction contributions of neural networks and empirical correlations. In doing so it provides further insight to the underlying data.Cited as: Wood, D.A., Choubineh, A. Reliable predictions of oil formation volume factor based on transparent and auditable machine learning approaches. Advances in Geo-Energy Research, 2019, 3(3): 225-241, doi: 10.26804/ager.2019.03.0

Estimation of minimum miscibility pressure of varied gas compositions and reservoir crude oil over a wide range of conditions using an artificial neural network model

Minimum miscibility pressure (MMP) is a key variable for monitoring miscibility between reservoir fluid and injection gas. Experimental and non-experimental methods are used to estimate MMP. Available miscibility correlations attempt to predict the minimum miscibility pressure for a specific type of gas. Here an artificial neural network (ANN) model is applied to a dataset involving 251 data records from around the world in a novel way to estimate the gas-crude oil MMP for a wide range of injected gases and crude oil compositions. This approach is relevant to sequestration projects in which injected gas compositions might vary significantly. The model is correlated with the reservoir temperature, concentrations of volatile (C1 and N2) and intermediate (C2, C3, C4, CO2 and H2S) fractions in the oil (Vol/Inter), C5+ molecular weight fractions in the oil and injected gas specific gravity. A key benefit of the ANN model is that MMP can be determined with reasonable accuracy for a wide range of oil and gas compositions. Statistical comparison of predictions shows that the developed ANN model yields better predictions than empirical-correlation methods. The ANN model predictions achieve a mean absolute percentage error of 13.46%, root mean square error of 3.6 and Pearson's correlation coefficient of 0.95. Sensitivity analysis reveals that injected gas specific gravity and temperature are the most important factors to consider when establishing appropriate miscible injection conditions. Among the available published correlations, the Yellig and Metcalfe correlation demonstrates good prediction performance, but it is not as accurate as the developed ANN model.Cited as: Choubineh, A., Helalizadeh, A., Wood, D.A. Estimation of minimum miscibility pressure of varied gas compositions and reservoir crude oil over a wide range of conditions using an artificial neural network model. Advances in Geo-Energy Research, 2019, 3(1): 52-66, doi: 10.26804/ager.2019.01.0

Estimation of minimum miscibility pressure of varied gas compositions and reservoir crude oil over a wide range of conditions using an artificial neural network model

Minimum miscibility pressure (MMP) is a key variable for monitoring miscibility between reservoir fluid and injection gas. Experimental and non-experimental methods are used to estimate MMP. Available miscibility correlations attempt to predict the minimum miscibility pressure for a specific type of gas. Here an artificial neural network (ANN) model is applied to a dataset involving 251 data records from around the world in a novel way to estimate the gas-crude oil MMP for a wide range of injected gases and crude oil compositions. This approach is relevant to sequestration projects in which injected gas compositions might vary significantly. The model is correlated with the reservoir temperature, concentrations of volatile (C1 and N2) and intermediate (C2, C3, C4, CO2 and H2S) fractions in the oil (Vol/Inter), C5+ molecular weight fractions in the oil and injected gas specific gravity. A key benefit of the ANN model is that MMP can be determined with reasonable accuracy for a wide range of oil and gas compositions. Statistical comparison of predictions shows that the developed ANN model yields better predictions than empirical-correlation methods. The ANN model predictions achieve a mean absolute percentage error of 13.46%, root mean square error of 3.6 and Pearson's correlation coefficient of 0.95. Sensitivity analysis reveals that injected gas specific gravity and temperature are the most important factors to consider when establishing appropriate miscible injection conditions. Among the available published correlations, the Yellig and Metcalfe correlation demonstrates good prediction performance, but it is not as accurate as the developed ANN model

The impacts of gas impurities on the minimum miscibility pressure of injected CO2-rich gas–crude oil systems and enhanced oil recovery potential

Abstract An effective parameter in the miscible-CO2 enhanced oil recovery procedure is the minimum miscibility pressure (MMP) defined as the lowest pressure that the oil in place and the injected gas into reservoir achieve miscibility at a given temperature. Flue gases released from power plants can provide an available source of CO2, which would otherwise be emitted to the atmosphere, for injection into a reservoir. However, the costs related to gas extraction from flue gases is potentially high. Hence, greater understanding the role of impurities in miscibility characteristics between CO2 and reservoir fluids helps to establish which impurities are tolerable and which are not. In this study, we simulate the effects of the impurities nitrogen (N2), methane (C1), ethane (C2) and propane (C3) on CO2 MMP. The simulation results reveal that, as an impurity, nitrogen increases CO2–oil MMP more so than methane. On the other hand, increasing the propane (C3) content can lead to a significant decrease in CO2 MMP, whereas varying the concentrations of ethane (C2) does not have a significant effect on the minimum miscibility pressure of reservoir crude oil and CO2 gas. The novel relationships established are particularly valuable in circumstances where MMP experimental data are not available

Applying Monte Carlo Dropout to Quantify the Uncertainty of Skip Connection-Based Convolutional Neural Networks Optimized by Big Data

Although Deep Learning (DL) models have been introduced in various fields as effective prediction tools, they often do not care about uncertainty. This can be a barrier to their adoption in real-world applications. The current paper aims to apply and evaluate Monte Carlo (MC) dropout, a computationally efficient approach, to investigate the reliability of several skip connection-based Convolutional Neural Network (CNN) models while keeping their high accuracy. To do so, a high-dimensional regression problem is considered in the context of subterranean fluid flow modeling using 376,250 generated samples. The results demonstrate the effectiveness of MC dropout in terms of reliability with a Standard Deviation (SD) of 0.012–0.174, and of accuracy with a coefficient of determination (R2) of 0.7881–0.9584 and Mean Squared Error (MSE) of 0.0113–0.0508, respectively. The findings of this study may contribute to the distribution of pressure in the development of oil/gas fields.</jats:p

An innovative application of deep learning in multiscale modeling of subsurface fluid flow Reconstructing the basis functions of the mixed Generalized Multiscale Finite Element Method (GMsFEM)

In multiscale modeling of subsurface fluid flow in heterogeneous porous media, standard polynomial basis functions are replaced by multiscale basis functions. For instance, to produce such functions in the mixed Generalized Multiscale Finite Element Method (mixed GMsFEM), a number of Partial Differential Equations (PDEs) must be solved, which requires a considerable overhead. Thus, it makes sense to replace PDE solvers with data-driven methods, given their great capabilities and general acceptance in the recent decades. Convolutional Neural Networks (CNNs) automatically perform feature engineering, and they also need fewer parameters via defining two-dimensional convolutional filters without reducing the quality of models. This is why four distinct CNN models were developed to predict four different multiscale basis functions for the mixed GMsFEM in the present study. These models were applied to 249,375 samples, with the permeability field as the only input. The statistical results indicate that the AMSGrad optimization algorithm with a coefficient of determination (R2) of 0.8434–0.9165 and Mean Squared Error (MSE) of 0.0078–0.0206 performs slightly better than Adam with an R2 of 0.8328–0.9049 and MSE of 0.0109–0.0261. Graphically, all models precisely follow the observed trend in each coarse block. This work could contribute to the distribution of pressure and velocity in the development of oil/gas fields. Looking at this work as an image (matrix)-to-image (matrix) regression problem, the constructed data-driven-based models may have applications beyond reservoir engineering, such as hydrogeology and rock mechanics

Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset

Machine Learning (ML) and/or Deep Learning (DL) methods can be used to predict fluid flow in porous media, as a suitable replacement for classical numerical approaches. Such data-driven approaches attempt to learn mappings between finite-dimensional Euclidean spaces. A novel neural framework, named Fourier Neural Operator (FNO), has been recently developed to act on infinite-dimensional spaces. A high proportion of the research available on the FNO has focused on problems with large-shape data. Furthermore, most published studies apply the FNO method to existing datasets. This paper applies and evaluates FNO to predict pressure distribution over a small, specified shape-data problem using 1700 Finite Element Method (FEM) generated samples, from heterogeneous permeability fields as the input. Considering FEM-calculated outputs as the true values, the configured FNO model provides superior prediction performance to that of a Convolutional Neural Network (CNN) in terms of statistical error assessment based on the coefficient of determination (R2) and Mean Squared Error (MSE). Sensitivity analysis considering a range of FNO configurations reveals that the most accurate model is obtained using modes=15 and width=100. Graphically, the FNO model precisely follows the observed trend in each porous medium evaluated. There is potential to further improve the FNO’s performance by including physics constraints in its network configuration