Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

A General Spatio-Temporal Clustering-Based Non-local Formulation for Multiscale Modeling of Compartmentalized Reservoirs

Representing the reservoir as a network of discrete compartments with neighbor and non-neighbor connections is a fast, yet accurate method for analyzing oil and gas reservoirs. Automatic and rapid detection of coarse-scale compartments with distinct static and dynamic properties is an integral part of such high-level reservoir analysis. In this work, we present a hybrid framework specific to reservoir analysis for an automatic detection of clusters in space using spatial and temporal field data, coupled with a physics-based multiscale modeling approach. In this work a novel hybrid approach is presented in which we couple a physics-based non-local modeling framework with data-driven clustering techniques to provide a fast and accurate multiscale modeling of compartmentalized reservoirs. This research also adds to the literature by presenting a comprehensive work on spatio-temporal clustering for reservoir studies applications that well considers the clustering complexities, the intrinsic sparse and noisy nature of the data, and the interpretability of the outcome. Keywords: Artificial Intelligence; Machine Learning; Spatio-Temporal Clustering; Physics-Based Data-Driven Formulation; Multiscale Modelin

Population-based algorithms for improved history matching and uncertainty quantification of Petroleum reservoirs

In modern field management practices, there are two important steps that shed light on a multimillion dollar investment. The first step is history matching where the simulation model is calibrated to reproduce the historical observations from the field. In this inverse problem, different geological and petrophysical properties may provide equally good history matches. Such diverse models are likely to show different production behaviors in future. This ties the history matching with the second step, uncertainty quantification of predictions. Multiple history matched models are essential for a realistic uncertainty estimate of the future field behavior. These two steps facilitate decision making and have a direct impact on technical and financial performance of oil and gas companies. Population-based optimization algorithms have been recently enjoyed growing popularity for solving engineering problems. Population-based systems work with a group of individuals that cooperate and communicate to accomplish a task that is normally beyond the capabilities of each individual. These individuals are deployed with the aim to solve the problem with maximum efficiency. This thesis introduces the application of two novel population-based algorithms for history matching and uncertainty quantification of petroleum reservoir models. Ant colony optimization and differential evolution algorithms are used to search the space of parameters to find multiple history matched models and, using a Bayesian framework, the posterior probability of the models are evaluated for prediction of reservoir performance. It is demonstrated that by bringing latest developments in computer science such as ant colony, differential evolution and multiobjective optimization, we can improve the history matching and uncertainty quantification frameworks. This thesis provides insights into performance of these algorithms in history matching and prediction and develops an understanding of their tuning parameters. The research also brings a comparative study of these methods with a benchmark technique called Neighbourhood Algorithms. This comparison reveals the superiority of the proposed methodologies in various areas such as computational efficiency and match quality

Assisted history matching using pattern recognition technology

Reservoir simulation and modeling is utilized throughout field development in different capacities. Sensitivity analysis, history matching, operations optimization and uncertainty assessment are the conventional analyses in full field model studies. Realistic modeling of the complexities of a reservoir requires a large number of grid blocks. As the complexity of a reservoir increases and consequently the number of grid blocks, so does the time required to accomplish the abovementioned tasks.;This study aims to examine the application of pattern recognition technologies to improve the time and efforts required for completing successful history matching projects. The pattern recognition capabilities of Artificial Intelligence and Data Mining (AI&DM;) techniques are used to develop a Surrogate Reservoir Model (SRM) and use it as the engine to drive the history matching process. SRM is a prototype of the full field reservoir simulation model that runs in fractions of a second. SRM is built using a small number of geological realizations.;To accomplish the objectives of this work, a three step process was envisioned:;• Part one, a proof of concept study: The goal of first step was to prove that SRM is able to substitute the reservoir simulation model in a history matching project. In this part, the history match was accomplished by tuning only one property (permeability) throughout the reservoir.;• Part two, a feasibility study: This step aimed to study the feasibility of SRM as an effective tool to solve a more complicated history matching problem, particularly when the degrees of uncertainty in the reservoir increase. Therefore, the number of uncertain reservoir properties increased to three properties (permeability, porosity, and thickness). The SRM was trained, calibrated, and validated using a few geological realizations of the base reservoir model. In order to complete an automated history matching workflow, the SRM was coupled with a global optimization algorithm called Differential Evolution (DE). DE optimization method is considered as a novel and robust optimization algorithm from the class of evolutionary algorithm methods.;• Part three, a real-life challenge: The final step was to apply the lessons learned in order to achieve the history match of a real-life problem. The goal of this part was to challenge the strength of SRM in a more complicated case study. Thus, a standard test reservoir model, known as PUNQ-S3 reservoir model in the petroleum engineering literature, was selected. The PUNQ-S3 reservoir model represents a small size industrial reservoir engineering model. This model has been formulated to test the ability of various methods in the history matching and uncertainty quantification. The surrogate reservoir model was developed using ten geological realizations of the model. The uncertain properties in this model are distributions of porosity, horizontal, and vertical permeability. Similar to the second part of this study, the DE optimization method was connected to the SRM to form an automated workflow in order to perform the history matching. This automated workflow is able to produce multiple realizations of the reservoir which match the past performance. The successful matches were utilized to quantify the uncertainty in the prediction of cumulative oil production.;The results of this study prove the ability of the surrogate reservoir models, as a fast and accurate tool, to address the practical issues of reservoir simulation models in the history matching workflow. Nevertheless, the achievements of this dissertation are not only aimed at the history matching procedure, but also benefit the other time-consuming operations in the reservoir management workflow (such as sensitivity analysis, production optimization, and uncertainty assessment)

Adaptive algorithms for history matching and uncertainty quantification

Numerical reservoir simulation models are the basis for many decisions in regard to predicting, optimising, and improving production performance of oil and gas reservoirs. History matching is required to calibrate models to the dynamic behaviour of the reservoir, due to the existence of uncertainty in model parameters. Finally a set of history matched models are used for reservoir performance prediction and economic and risk assessment of different development scenarios. Various algorithms are employed to search and sample parameter space in history matching and uncertainty quantification problems. The algorithm choice and implementation, as done through a number of control parameters, have a significant impact on effectiveness and efficiency of the algorithm and thus, the quality of results and the speed of the process. This thesis is concerned with investigation, development, and implementation of improved and adaptive algorithms for reservoir history matching and uncertainty quantification problems. A set of evolutionary algorithms are considered and applied to history matching. The shared characteristic of applied algorithms is adaptation by balancing exploration and exploitation of the search space, which can lead to improved convergence and diversity. This includes the use of estimation of distribution algorithms, which implicitly adapt their search mechanism to the characteristics of the problem. Hybridising them with genetic algorithms, multiobjective sorting algorithms, and real-coded, multi-model and multivariate Gaussian-based models can help these algorithms to adapt even more and improve their performance. Finally diversity measures are used to develop an explicit, adaptive algorithm and control the algorithm’s performance, based on the structure of the problem. Uncertainty quantification in a Bayesian framework can be carried out by resampling of the search space using Markov chain Monte-Carlo sampling algorithms. Common critiques of these are low efficiency and their need for control parameter tuning. A Metropolis-Hastings sampling algorithm with an adaptive multivariate Gaussian proposal distribution and a K-nearest neighbour approximation has been developed and applied

Strategies for real time reservoir management

Real–time reservoir management is developed to manage a shrinking labor force and rising demand on energy supply. This dissertation seeks good strategies for real–time reservoir management. First, two simulator–independent optimization algorithms are investigated: ensemble–based optimization (EnOpt) and bound optimization by quadratic approximation (BOBYQA). Multiscale regularization is applied to both to find appropriate frequencies for well control adjustment. Second, two gathered EnKF methods are proposed to save computational cost and reduce sampling error: gathered EnKF with a fixed gather size and adaptively gathered EnKF. Finally, oil price uncertainty is forecasted and quantified with three price forecasting models: conventional forecasting, bootstrap forecasting and sequential Gaussian simulation forecasting. The relative effect of oil price and its volatility on the optimization strategies are investigated. A number of key findings of this dissertation are: (a) if multiscale regularization is not used, EnOpt converges to a higher net present value (NPV) than BOBYQA—even though BOBYQA uses second order Hessian information whereas EnOpt uses first order gradients. BOBYQA performs comparably only if multiscale regularization is used. Multiscale regularization results in a higher optimized NPV with simpler well control strategies and converges in fewer iterations; (b) gathering observations not only reduces the sampling errors but also saves significant amount of computational cost. In addition, adaptively gathered EnKF is superior to gathered EnKF with a fixed gather size when the prior ensemble mean is not near the truth; (c) it is shown that a good oil price forecasting model can improve NPV by more than four percent, and (d) instability in oil prices also causes fluctuation in optimized well controls

Establishment of Landslide Groundwater Level Prediction Model Based on GA-SVM and Influencing Factor Analysis

The monitoring and prediction of the landslide groundwater level is a crucial part of landslide early warning systems. In this study, Tangjiao landslide in the Three Gorges Reservoir area (TGRA) in China was taken as a case study. Three groundwater level monitoring sensors were installed in different locations of the landslide. The monitoring data indicated that the fluctuation of groundwater level is significantly consistent with rainfall and reservoir level in time, but there is a lag. In addition, there is a spatial difference in the impact of reservoir levels on the landslide groundwater level. The data of two monitoring locations were selected for establishing the prediction model of groundwater. Combined with the qualitative and quantitative analysis, the influencing factors were selected, respectively, to establish the hybrid Genetic Algorithm-Support Vector Machine (GA-SVM) prediction model. The single-factor GA-SVM without considering influencing factors and the backpropagation neural network (BPNN) model were adopted to make comparisons. The results showed that the multi-factor GA-SVM performed the best, followed by multi-factor BPNN and single-factor GA-SVM. We found that the prediction accuracy can be improved by considering the influencing factor. The proposed GA-SVM model combines the advantages of each algorithm; it can effectively construct the response relationship between groundwater level fluctuations and influencing factors. Above all, the multi-factor GA-SVM is an effective method for the prediction of landslides groundwater in the TGRA