Parallel 3-D marine controlled-source electromagnetic modelling using high-order tetrahedral Nédélec elements

We present a parallel and high-order Nédélec finite element solution for the marine controlled-source electromagnetic (CSEM) forward problem in 3-D media with isotropic conductivity. Our parallel Python code is implemented on unstructured tetrahedral meshes, which support multiple-scale structures and bathymetry for general marine 3-D CSEM modelling applications. Based on a primary/secondary field approach, we solve the diffusive form of Maxwell’s equations in the low-frequency domain. We investigate the accuracy and performance advantages of our new high-order algorithm against a low-order implementation proposed in our previous work. The numerical precision of our high-order method has been successfully verified by comparisons against previously published results that are relevant in terms of scale and geological properties. A convergence study confirms that high-order polynomials offer a better trade-off between accuracy and computation time. However, the optimum choice of the polynomial order depends on both the input model and the required accuracy as revealed by our tests. Also, we extend our adaptive-meshing strategy to high-order tetrahedral elements. Using adapted meshes to both physical parameters and high-order schemes, we are able to achieve a significant reduction in computational cost without sacrificing accuracy in the modelling. Furthermore, we demonstrate the excellent performance and quasi-linear scaling of our implementation in a state-of-the-art high-performance computing architecture.This project has received funding from the European Union's Horizon 2020 programme under the Marie Sklodowska-Curie grant agreement No. 777778. Furthermore, the research leading to these results has received funding from the European Union's Horizon 2020 programme under the ChEESE Project (https://cheese-coe.eu/ ), grant agreement No. 823844. In addition, the authors would also like to thank the support of the Ministerio de Educación y Ciencia (Spain) under Projects TEC2016-80386-P and TIN2016-80957-P. The authors would like to thank the Editors-in-Chief and to both reviewers, Dr. Martin Cuma and Dr. Raphael Rochlitz, for their valuable comments and suggestions which helped to improve the quality of the manuscript. This work benefited from the valuable suggestions, comments, and proofreading of Dr. Otilio Rojas (BSC). Last but not least, Octavio Castillo-Reyes thanks Natalia Gutierrez (BSC) for her support in CSEM modeling with BSIT.Peer ReviewedPostprint (author's final draft

PETGEM: A parallel code for 3D CSEM forward modeling using edge finite elements

We present the capabilities and results of the Parallel Edge-based Tool for Geophysical Electromagnetic modeling (PETGEM), as well as the physical and numerical foundations upon which it has been developed. PETGEM is an open-source and distributed parallel Python code for fast and highly accurate modeling of 3D marine controlled-source electromagnetic (3D CSEM) problems. We employ the N\'ed\'elec Edge Finite Element Method (EFEM) which offers a good trade-off between accuracy and number of degrees of freedom, while naturally supporting unstructured tetrahedral meshes. We have particularised this new modeling tool to the 3D CSEM problem for infinitesimal point dipoles asumming arbitrarily isotropic media for low-frequencies approximations. In order to avoid source-singularities, PETGEM solves the frequency-domain Maxwell's equations of the secondary electric field, and the primary electric field is calculated analytically for homogeneous background media. We assess the PETGEM accuracy using classical tests with known analytical solutions as well as recent published data of real life geological scenarios. This assessment proves that this new modeling tool reproduces expected accurate solutions in the former tests, and its flexibility on realistic 3D electromagnetic problems. Furthermore, an automatic mesh adaptation strategy for a given frequency and specific source position is presented. We also include a scalability study based on fundamental metrics for high-performance computing (HPC) architectures.Comment: \c{opyright} 2018. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/ This project has received funding from the EC-H2020 under the Marie Sklodowska-Curie grant agreement No. 644202, and from the EC-H2020 under the HPC4E Project, grant agreement No. 68977

custEM: Customizable finite-element simulation of complex controlled-source electromagnetic data

We have developed the open-source toolbox custEM (customizable electromagnetic modeling) for the simulation of complex 3D controlled-source electromagnetic (CSEM) problems. It is based on the open-source finite-element library FEniCS, which supports tetrahedral meshes, multiprocessing, higher order polynomials, and anisotropy. We use multiple finite-element approaches to solve the time-harmonic Maxwell equations, which are based on total or secondary electric field and gauged potential formulations. In addition, we develop a secondary magnetic field formulation, showing superior performance if only magnetic fields are required. Using Nédélec basis functions, we robustly incorporate the current density on the edges of the mesh for the total field formulations. The latter enable modeling of CSEM problems taking topography into account. We evaluate semianalytical 1D layered-earth solutions with the pyhed library, supporting arbitrary configurations of dipole or loop sources for secondary field calculations. All system matrices have been modified to be symmetric and solved in parallel with the direct solver MUMPS. Aside from the finite-element kernel, mesh generation, interpolation, and visualization modules have been implemented to simplify and automate the modeling workflow. We prove the capability of custEM, including validation against analytic-solutions, crossvalidation of all implemented approaches, and results for a model with 3D topography with four examples. The object-oriented implementation allows for customizable modifications and additions or to use only submodules designed for special tasks, such as mesh generation or matrix assembly. Therefore, the toolbox is suitable for crossvalidation with other codes and as the basis for developing 3D inversion routines

Evaluation of three approaches for simulating 3D time-domain electromagnetic data

We implemented and compared the implicit Euler time-stepping approach, the inverse Fourier Transform-based approach, and the Rational Arnoldi method for simulating 3D transient electromagnetic data. We utilize the finite-element method with unstructured tetrahedral meshes for the spatial discretization supporting irregular survey geometries and anisotropic material parameters. Both, switch-on and switch-off current waveforms, can be used in combination with direct current solutions of Poisson problems as initial conditions. Moreover, we address important topics such as the incorporation of source currents and opportunities to simulate impulse as well as step response magnetic field data with all approaches for supporting a great variety of applications. Three examples ranging from simple to complex real-world geometries and validations against external codes provide insight into the numerical accuracy, computational performance, and unique characteristics of the three applied methods. We further present an application of logarithmic Fourier transforms to convert transient data into the frequency domain. We made all approaches available in the open-source Python toolbox custEM, which previously supported only frequency-domain electromagnetic data. The object-oriented software implementation is suited for further elaboration on distinct modeling topics and the presented examples can serve for benchmarking other codes

Meshing strategies for 3d geo-electromagnetic modeling in the presence of metallic infrastructure

In 3D geo-electromagnetic modeling, an adequate discretisation of the modeling domain is crucial to obtain accurate forward responses and reliable inversion results while reducing the computational cost. This paper investigates the mesh design for subsurface models, including steel-cased wells, which is relevant for many exploration settings but still remains a numerically challenging task. Applying a goal-oriented mesh refinement technique and subsequent calculations with the high-order edge finite element method, simulations of 3D controlled-source electromagnetic models in the presence of metallic infrastructure are performed. Two test models are considered, each needing a distinct version of approximation methods to incorporate the conductive steel casings of the included wells. The influence of mesh quality, goal-oriented meshing, and high-order approximations on problem sizes, computational cost, and accuracy of electromagnetic responses is investigated. The main insights of our work are: (a) the applied numerical schemes can mitigate the computational burden of geo-electromagnetic modeling in the presence of steel artifacts; (b) investigating the processes driving the meshing of models with embedded metallic infrastructures can lead to adequate strategies to deal with the inversion of such electromagnetic data sets. Based on the modeling results and analyses conducted, general recommendations for modeling strategies are proposed when performing simulations for challenging steel infrastructure scenarios.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The work of O.C-R. has received funding from the Ministerio de Educación y Ciencia (Spain) under Project TED2021-131882B-C42.The code development of P.R. has been financed by the Smart Exploration project. Smart Exploration has received funding from the European Union’s Horizon 2020 Framework Programme under grant agreement N∘ 775971. The computations were enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at UPPMAX partially funded by the Swedish Research Council through grant agreement N∘ SNIC 2021/22-883.Peer ReviewedPostprint (published version

HPC geophysical electromagnetics: a synthetic VTI model with complex bathymetry

We introduce a new synthetic marine model for 3D controlled-source electromagnetic method (CSEM) surveys. The proposed model includes relevant features for the electromagnetic geophysical community such as large conductivity contrast with vertical transverse isotropy and a complex bathymetry profile. In this paper, we present the experimental setup and several 3D CSEM simulations in the presence of a resistivity unit denoting a hydrocarbon reservoir. We employ a parallel and high-order vector finite element routine to perform the CSEM simulations. By using tailored meshes, several scenarios are simulated to assess the influence of the reservoir unit presence on the electromagnetic responses. Our numerical assessment confirms that resistivity unit strongly influences the amplitude and phase of the electromagnetic measurements. We investigate the code performance for the solution of fundamental frequencies on high-performance computing architectures. Here, excellent performance ratios are obtained. Our benchmark model and its modeling results are developed under an open-source scheme that promotes easy access to data and reproducible solutions.The work of O.C-R., conducted in the frame of PIXIL project, has been 65% cofinanced by the European Regional Development Fund (ERDF) through the Interreg V-A SpainFrance-Andorra program (POCTEFA2014-2020). BSC authors have received funding from the European Union’s Horizon 2020 programme under the Marie Sklodowska-Curie grant agreement N◦ 777778. Furthermore, the development of PETGEM has received funding from the European Union’s Horizon 2020 programme, grant agreement N◦ 828947, and from the Mexican Department of Energy, CONACYT-SENER Hidrocarburos grant agreement N◦ B-S-69926.Peer ReviewedPostprint (published version

3D Magnetotelluric Modeling Using High-Order Tetrahedral Nédélec Elements on Massively Parallel Computing Platforms

We present a routine for 3D magnetotelluric (MT) modeling based upon high-order edge finite element method (HEFEM), tailored and unstructured tetrahedral meshes, and high-performance computing (HPC). This implementation extends the PETGEM modeller capabilities, initially developed for active-source electromagnetic methods in frequency-domain. We assess the accuracy, robustness, and performance of the code using a set of reference models developed by the MT community in well-known reported workshops. The scale and geological properties of these 3D MT setups are challenging, making them ideal for addressing a rigorous validation. Our numerical assessment proves that this new algorithm can produce the expected solutions for arbitrarily 3D MT models. Also, our extensive experimental results reveal four main insights: (1) high-order discretizations in conjunction with tailored meshes can offer excellent accuracy; (2) a rigorous mesh design based on the skin-depth principle can be beneficial for the solution of the 3D MT problem in terms of numerical accuracy and run-time; (3) high-order polynomial basis functions achieve better speed-up and parallel efficiency ratios than low-order polynomial basis functions on cutting-edge HPC platforms; (4) a triple helix approach based on HEFEM, tailored meshes, and HPC can be extremely competitive for the solution of realistic and complex 3D MT models and geophysical electromagnetics in general.This project has been 65% cofinanced by the European Regional Development Fund (ERDF) through the Interreg V-A Spain–France– Andorra program (POCTEFA2014-2020). POCTEFA aims to reinforce the economic and social integration of the French–Spanish–Andorran border. Its support is focused on developing economic, social and environmental cross-border activities through joint strategies favoring sustainable territorial development. BSC authors received funding from the European Union’s Horizon 2020 programme, grant agreement N◦828947 and N◦777778, and from the Mexican Department of Energy, CONACYT-SENER Hidrocarburos grant agreement N◦B-S-69926

3D magnetotelluric modeling using high-order tetrahedral Nédélec elements on massively parallel computing platforms

We present a routine for 3D magnetotelluric (MT) modeling based upon high-order edge finite element method (HEFEM), tailored and unstructured tetrahedral meshes, and high-performance computing (HPC). This implementation extends the PETGEM modeller capabilities, initially developed for active-source electromagnetic methods in frequency-domain. We assess the accuracy, robustness, and performance of the code using a set of reference models developed by the MT community in well-known reported workshops. The scale and geological properties of these 3D MT setups are challenging, making them ideal for addressing a rigorous validation. Our numerical assessment proves that this new algorithm can produce the expected solutions for arbitrarily 3D MT models. Also, our extensive experimental results reveal four main insights: (1) high-order discretizations in conjunction with tailored meshes can offer excellent accuracy; (2) a rigorous mesh design based on the skin-depth principle can be beneficial for the solution of the 3D MT problem in terms of numerical accuracy and run-time; (3) high-order polynomial basis functions achieve better speed-up and parallel efficiency ratios than low-order polynomial basis functions on cutting-edge HPC platforms; (4) a triple helix approach based on HEFEM, tailored meshes, and HPC can be extremely competitive for the solution of realistic and complex 3D MT models and geophysical electromagnetics in general

Land CSEM Simulations and Experimental Test Using Metallic Casing in a Geothermal Exploration Context: Vallès Basin (NE Spain) Case Study

Controlled-source electromagnetic (CSEM) measurements are complementary data for magnetotelluric (MT) characterization although its methodology on land is not sufficiently developed and tested as in marine environments. Acquiring expertise in CSEM is crucial for surveys in places where MT cannot be performed due to high levels of cultural noise. To acquire that expertise, we perform CSEM experiments in the Vallès fault [Northeast (NE), Spain], where MT results have been satisfactory and allow us to verify the CSEM results. The Vallès basin is relevant for potential heat generation because of the presence of several geothermal anomalies and its nearby location in urban areas. In this article, we present the experimental setup for that region, a 2-D joint MT+CSEM inverse model, several 3-D CSEM simulations in the presence of metallic casing, and its comparison with real data measurements. We employ a parallel and high-order vector finite element algorithm to discretize the governing equations. By using an adapted meshing strategy, different scenarios are simulated to study the influence of the source position/direction and the conductivity model in a metallic casing presence. An excellent agreement between the simulated data and analytical/real field data demonstrates the feasibility of study metallic structures in realistic configurations. Our numerical results confirm that metallic casing strongly influences electromagnetic (EM) responses, making surface measurements more sensitive to resistivity variations near the metallic structure. It could be beneficial getting higher signal-to-noise ratios and sensitivity to deep targets. However, such a casing effect depends on the input model (e.g., conductivity contrasts, frequency, and geometry)

A 3D Time Domain CSEM Forward Modeling Code using custEM and FEniCS

A new 3D time domain CSEM forward modeling code TDcustEM built upon the recently published open-source frequency domain code custEM and the open-source finite element toolbox FEniCS is presented. The transformation of the frequency domain data provided by custEM is performed by a Fast Hankel Transform using 80 digital filter coefficients. 3D edge-based tetrahedral meshes generated by TetGen facilitate the calculation of syn- thetic data using topography, arbitrary source geometries and complex subsurface structures. To ensure precision and reliability of the new algorithm, calculated results of different CSEM setups and electromagnetic field components are cross-validated against analytic solutions as well as 1D and 3D time domain modeling codes. Certain modeling studies are conducted regarding possible interpolation and extrapola- tion techniques to reduce the number of necessary frequencies and therefore the compu- tational runtime, which is still an issue of convolutional time domain CSEM approaches. Additional modeling studies showed the importance of precise receiver positioning for measuring the horizontal components of the time derivative of the magnetic field. As the present thesis is embedded in the Collaborative Research Centre 806 – Our Way to Europe, three sedimentary deposits in the East African Rift Valley were subject to mul- tidimensional TEM surveys in the framework of this project. Common 1D (EMUPLUS) as well as laterally and spatially constrained (AarhusInv) in- version techniques were applied to the TEM field data. Sediment thicknesses as well as stratigraphic sequences of the investigated sedimentary basins were derived. An extensive 3D modeling study of one of the target areas representing a volcanically-formed basin including topography was performed using the newly developed TDcustEM code