Parallel 3-D marine controlled-source electromagnetic modelling using high-order tetrahedral Nédélec elements

We present a parallel and high-order Nédélec finite element solution for the marine controlled-source electromagnetic (CSEM) forward problem in 3-D media with isotropic conductivity. Our parallel Python code is implemented on unstructured tetrahedral meshes, which support multiple-scale structures and bathymetry for general marine 3-D CSEM modelling applications. Based on a primary/secondary field approach, we solve the diffusive form of Maxwell’s equations in the low-frequency domain. We investigate the accuracy and performance advantages of our new high-order algorithm against a low-order implementation proposed in our previous work. The numerical precision of our high-order method has been successfully verified by comparisons against previously published results that are relevant in terms of scale and geological properties. A convergence study confirms that high-order polynomials offer a better trade-off between accuracy and computation time. However, the optimum choice of the polynomial order depends on both the input model and the required accuracy as revealed by our tests. Also, we extend our adaptive-meshing strategy to high-order tetrahedral elements. Using adapted meshes to both physical parameters and high-order schemes, we are able to achieve a significant reduction in computational cost without sacrificing accuracy in the modelling. Furthermore, we demonstrate the excellent performance and quasi-linear scaling of our implementation in a state-of-the-art high-performance computing architecture.This project has received funding from the European Union's Horizon 2020 programme under the Marie Sklodowska-Curie grant agreement No. 777778. Furthermore, the research leading to these results has received funding from the European Union's Horizon 2020 programme under the ChEESE Project (https://cheese-coe.eu/ ), grant agreement No. 823844. In addition, the authors would also like to thank the support of the Ministerio de Educación y Ciencia (Spain) under Projects TEC2016-80386-P and TIN2016-80957-P. The authors would like to thank the Editors-in-Chief and to both reviewers, Dr. Martin Cuma and Dr. Raphael Rochlitz, for their valuable comments and suggestions which helped to improve the quality of the manuscript. This work benefited from the valuable suggestions, comments, and proofreading of Dr. Otilio Rojas (BSC). Last but not least, Octavio Castillo-Reyes thanks Natalia Gutierrez (BSC) for her support in CSEM modeling with BSIT.Peer ReviewedPostprint (author's final draft

Tunable plasmonic resonances in highly porous nano-bamboo Si-Au superlattice-type thin films

We report on fabrication of spatially-coherent columnar plasmonic nanostructure superlattice-type thin films with high porosity and strong optical anisotropy using glancing angle deposition. Subsequent and repeated depositions of silicon and gold lead to nanometer-dimension subcolumns with controlled lengths. The superlattice-type columns resemble bamboo structures where smaller column sections of gold form junctions sandwiched between larger silicon column sections ("nano-bamboo"). We perform generalized spectroscopic ellipsometry measurements and finite element method computations to elucidate the strongly anisotropic optical properties of the highly-porous nano-bamboo structures. The occurrence of a strongly localized plasmonic mode with displacement pattern reminiscent of a dark quadrupole mode is observed in the vicinity of the gold subcolumns. We demonstrate tuning of this quadrupole-like mode frequency within the near-infrared spectral range by varying the geometry of the nano-bamboo structure. In addition, coupled-plasmon-like and inter-band transition-like modes occur in the visible and ultra-violet spectral regions, respectively. We elucidate an example for the potential use of the nano-bamboo structures as a highly porous plasmonic sensor with optical read out sensitivity to few parts-per-million solvent levels in water

HPC geophysical electromagnetics: a synthetic VTI model with complex bathymetry

We introduce a new synthetic marine model for 3D controlled-source electromagnetic method (CSEM) surveys. The proposed model includes relevant features for the electromagnetic geophysical community such as large conductivity contrast with vertical transverse isotropy and a complex bathymetry profile. In this paper, we present the experimental setup and several 3D CSEM simulations in the presence of a resistivity unit denoting a hydrocarbon reservoir. We employ a parallel and high-order vector finite element routine to perform the CSEM simulations. By using tailored meshes, several scenarios are simulated to assess the influence of the reservoir unit presence on the electromagnetic responses. Our numerical assessment confirms that resistivity unit strongly influences the amplitude and phase of the electromagnetic measurements. We investigate the code performance for the solution of fundamental frequencies on high-performance computing architectures. Here, excellent performance ratios are obtained. Our benchmark model and its modeling results are developed under an open-source scheme that promotes easy access to data and reproducible solutions.The work of O.C-R., conducted in the frame of PIXIL project, has been 65% cofinanced by the European Regional Development Fund (ERDF) through the Interreg V-A SpainFrance-Andorra program (POCTEFA2014-2020). BSC authors have received funding from the European Union’s Horizon 2020 programme under the Marie Sklodowska-Curie grant agreement N◦ 777778. Furthermore, the development of PETGEM has received funding from the European Union’s Horizon 2020 programme, grant agreement N◦ 828947, and from the Mexican Department of Energy, CONACYT-SENER Hidrocarburos grant agreement N◦ B-S-69926.Peer ReviewedPostprint (published version

HPC and edge elements for geophysical electromagnetic problems: an overview

In Finite Element Methods for solving electromagnetic problems, the use of Nédélec Elements has become very popular. In fact, Nédélec Elements are often said to be a cure to many difficulties that are encountered, particularly eliminating spurious solutions, and are claimed to yield accurate results. In this paper, we present our first steps in developing a Nédélec Elements code for simulation of geophysical electromagnetic problems and first ideas on how implement the key issues of Edge Elements in an efficient way on HPC platforms

HPC and edge elements for geophysical electromagnetic problems: an overview

In Finite Element Methods for solving electromagnetic problems, the use of Nédélec Elements has become very popular. In fact, Nédélec Elements are often said to be a cure to many difficulties that are encountered, particularly eliminating spurious solutions, and are claimed to yield accurate results. In this paper, we present our first steps in developing a Nédélec Elements code for simulation of geophysical electromagnetic problems and first ideas on how implement the key issues of Edge Elements in an efficient way on HPC platforms

custEM: Customizable finite-element simulation of complex controlled-source electromagnetic data

We have developed the open-source toolbox custEM (customizable electromagnetic modeling) for the simulation of complex 3D controlled-source electromagnetic (CSEM) problems. It is based on the open-source finite-element library FEniCS, which supports tetrahedral meshes, multiprocessing, higher order polynomials, and anisotropy. We use multiple finite-element approaches to solve the time-harmonic Maxwell equations, which are based on total or secondary electric field and gauged potential formulations. In addition, we develop a secondary magnetic field formulation, showing superior performance if only magnetic fields are required. Using Nédélec basis functions, we robustly incorporate the current density on the edges of the mesh for the total field formulations. The latter enable modeling of CSEM problems taking topography into account. We evaluate semianalytical 1D layered-earth solutions with the pyhed library, supporting arbitrary configurations of dipole or loop sources for secondary field calculations. All system matrices have been modified to be symmetric and solved in parallel with the direct solver MUMPS. Aside from the finite-element kernel, mesh generation, interpolation, and visualization modules have been implemented to simplify and automate the modeling workflow. We prove the capability of custEM, including validation against analytic-solutions, crossvalidation of all implemented approaches, and results for a model with 3D topography with four examples. The object-oriented implementation allows for customizable modifications and additions or to use only submodules designed for special tasks, such as mesh generation or matrix assembly. Therefore, the toolbox is suitable for crossvalidation with other codes and as the basis for developing 3D inversion routines

libEMM: A fictious wave domain 3D CSEM modelling library bridging sequential and parallel GPU implementation

This paper delivers a software -- libEMM -- for 3D controlled-source electromagnetics (CSEM) modelling in fictitious wave domain, based on the newly developed high-order finite-difference time-domain (FDTD) method on non-uniform grid. The numerical simulation can be carried out over a number of parallel processors using MPI-based high performance computing architecture. The FDTD kernel coded in C has been parallelized with OpenMP for speedup using local shared memory. In addition, the software features a GPU implementation of the same algorithm based on CUDA programming language, which can be cross-validated and compared in terms of efficiency. A perspective of libEMM on the horizon is its application to 3D CSEM inversion in land and marine environment

3D magnetotelluric modeling using high-order tetrahedral Nédélec elements on massively parallel computing platforms

We present a routine for 3D magnetotelluric (MT) modeling based upon high-order edge finite element method (HEFEM), tailored and unstructured tetrahedral meshes, and high-performance computing (HPC). This implementation extends the PETGEM modeller capabilities, initially developed for active-source electromagnetic methods in frequency-domain. We assess the accuracy, robustness, and performance of the code using a set of reference models developed by the MT community in well-known reported workshops. The scale and geological properties of these 3D MT setups are challenging, making them ideal for addressing a rigorous validation. Our numerical assessment proves that this new algorithm can produce the expected solutions for arbitrarily 3D MT models. Also, our extensive experimental results reveal four main insights: (1) high-order discretizations in conjunction with tailored meshes can offer excellent accuracy; (2) a rigorous mesh design based on the skin-depth principle can be beneficial for the solution of the 3D MT problem in terms of numerical accuracy and run-time; (3) high-order polynomial basis functions achieve better speed-up and parallel efficiency ratios than low-order polynomial basis functions on cutting-edge HPC platforms; (4) a triple helix approach based on HEFEM, tailored meshes, and HPC can be extremely competitive for the solution of realistic and complex 3D MT models and geophysical electromagnetics in general