Coster, D.

Hoenen, O.

Lakhlili, J.

Luk, O.

Scott, B.

Toussaint, U.

Veen, L.

Yudin, Y.

English

MPG.PuRe

Performing Validation, Verification, and Sensitivity Analysis on MultiscaleFusion Plasma Simulations with the VECMA ToolkitO.O. Luk1, J. Lakhlili1, U. von Toussaint1, L. E. Veen2,Y. Yudin1, O. Hoenen1, B.D. Scott1, D.P. Coster11 Max-Planck-Institut für Plasmaphysik, Garching, Germany,2 Netherlands eScience Center, Science Park 140, 1098 XG, Amsterdam, The NetherlandsTo effectively simulate the dynamics of a fusion plasma, one must take into account thewidely disparate phenomena happening in both space and time. Namely, the micro-instabilitiesdeveloped in plasma turbulence can affect the overall transport and ultimately destroy con-finement. Hence, a component based approach is used to couple various single-scale modelstogether in a workflow. This multiscale fusion workflow MFW (see Figure 1) uses MUSCLE toconnect the transport, equilibrium and turbulence models together with a module that convertsfluxes into transport coefficients. While this workflow has provided insights into the overallplasma transport with turbulence taken into account [1], it needs to get verified, validated andits uncertainties quantified (e.g. analyze sensitivity) before any meaningful comparison betweensimulation results and experimental data can be carried out.Figure 1: Multiscale fusion workflow.The VECMA toolkit [2] is utilized to studyuncertainties in the workflow: it is composedof several components that aim to aid users inbringing Verification, Validation and Uncer-tainty Quantification (VVUQ) into their com-plex single-scale or multiscale models. Forexample, EasyVVUQ [4] is a python-basedlibrary that enables the VVUQ process in asimulation model. MUSCLE3 [5] helps cou-ple single-scale models together to form amultiscale workflow and advanced UQ can be applied to the workflow as well. We use thesecomponents to guide us through the VV and sensitivity analysis (SA) processes in the MFW.To validate the results coming from MFW simulations, we quantitatively compared the elec-tron and ion temperature, Te and Ti respectively, distributions coming from simulations and ex-perimental data of the ASDEX Upgrade tokamak1. The more well-known similarity measuressuch as Hellinger distance, Jensen-Shannon distance, and Wasserstein metric were utilized for1https://www.ipp.mpg.de/16195/asdex47th EPS Conference on Plasma Physics P2.1075such task, and they are available in 2 components of the toolkit: FabSim32 and/or EasyVVUQ.The validation results using these metrics can be found in [3].We derived a different measure: the compatibility measure can provide a quantitative com-parison between distributions that places more emphasis on the lower moments. The measureproduces a compatibility distance, or total weight wtot , with a value between 0 and 1:wtot = (1−w)(µ2−µ1)22(σ21 +σ22 )+(µ2−µ1)2+w(γ2− γ1)22(σ21 +σ22 )+(µ2−µ1)2 +(|γ1|+ |γ2|)2. (1)w is the weighting factor with a value also ranging between 0 and 1. µ , σ , and γ are the mean,standard deviation, and skewness of a distribution, respectively. The users of this metric can varyw accordingly. For example, we tested w = 0.05, 0.5, 0.95, and wtot with w = 0.05 agrees withour intuition the most. We will use w= 0.05 in the rest of this paper. The initial validation resultsusing both compatibility distance and Z-test can be found in [3] as well. These two measureswill be implemented and become available in the VECMA toolkit in the near future.Next we search for a systematic approach to interpret wtot values, i.e. at what value of wtotshall we consider 2 distributions as the same. After comparing various distributions, we classifythe values into 4 categories: 0 ≤ wtot < 0.45: same; 0.45 ≤ wtot < 0.6: marginally the same;0.6≤ wtot < 0.75: significantly different; and, 0.75≤ wtot ≤ 1: highly significantly different.Next we take the compatibility measure to compare two T distributions from simulation. Theway we obtained distributions at every flux-tube is we take output T values from within onesimulation second period, place them in a time bin, and then calculate µ , σ and γ . The resultingµ and µ±σ of the T distribution at every flux-tube, at each 1-s time bin are plotted in Figure 2).We then use the compatibility measure to determine when the plasma reaches a quasi-steadystate by comparing distributions from two consecutive time bins n and n−1, starting with n = 2(or 2-3s time bin). We found the distributions from 3-4s and 4-5s time bins at every flux-tube,for both Te and Ti, are qualitatively the same based on the established categories. Therefore, wedetermine the plasma reaches a quasi-steady state at 5s, and use T distributions from 4-5s timebin to compare with experiment.The ASDEX Upgrade experiments have certain restrictions and diagnostics that leads to fur-ther uncertainties involved in the measurements, in addition to the reported upper and lowervalues. This is especially so near the edge (ρ̂tor near 1.0). First, we took the data and treatedthe distributions as a split-Gaussian. To account for the further uncertainties, we expand σ toinclude 10% of µ to broaden the spread of the temperature distributions. Therefore we define2https://github.com/djgroen/FabSim347th EPS Conference on Plasma Physics P2.1075Figure 2: Te (left) and Ti (right) profiles at every simulation second.Figure 3: Te (left) and Ti (right) distributions from simulation and ASDEX Upgrade shot #36266(top), and their compatibility distance and Z-score (bottom). The solid, dashed and dotted greenlines at the top panels represent µ , µ±σ , and µ±σe f f of split Gaussian (experiment) distribu-tion, respectively.an effective standard deviation: σe f f = σ +0.1µ .We take the distributions obtained from 4-5s time bin in simulation and compare that to theexperiment, in particular shot #36266. The compatibility measure is shown in Figure 3, and itindicates the two distributions are qualitatively the same (0 ≤ wtot < 0.45) at every flux-tube.We also compare the simulation results with shot #36297, and it yields the same outcome.An issue we encountered is that, as shown in Figure 2, Te profile changes shape starting atthe 7-8 s time bin. We compared T distributions at this time bin to the experiment’s, and thecompatibility measure shows the two distributions are qualitatively the same. If we used thecompatibility measure to determine plasma’s quasi-steady state like we stated earlier, then wewould terminate the simulation too early (at 5s). We wouldn’t be able to capture any interestingphysics going on starting at 7s. Therefore, we will need to refine our method in the future.The verification process to the workflow can be achieved via level of refinement on modelparameters until quantities of interest (QoI) display asymptotic behavior. This approach hasbeen applied to a simple fusion workflow that simulates a cylindrical plasma with fixed density,and it is available in the VECMA Toolkit3 tutorial. UQ using polynomial chaos expansion3https://www.vecma-toolkit.eu/47th EPS Conference on Plasma Physics P2.1075(PCE) method was performed to this workflow with the help of EasyVVUQ. We varied 5 inputparameters and sought level of refinement to µ and σ of Te distribution and 1st Sobol indicesby scanning the value of PCE polynomial order, from 1 to 6. This test showed polynomial orderof 3 is sufficient to get converged results. [6]Finally, the turbulence model is computationally the most expensive single-scale model withinthe workflow. We want to apply SA to the model in the future to help reduce the number of var-ied inputs and therefore cut down the sample size and cost. In the meantime, Sobol SA wasperformed on the same UQ example from earlier. The 1st Sobol indices across the radial profileshow that, at the sixth PCE order, Te variance is most sensitive to the width of the heat sourcefunction, transport coefficient across the radial profile, and the edge Te value. [6]In conclusion, MFW uses component based approach and MUSCLE is to couple the single-scale models together into a workflow. We have made progress towards validation of the AS-DEX Upgrade tokamak simulations . We used EasyVVUQ and MUSCLE3 from the VECMAtoolkit to guide us in the VVUQ process. We quantitatively compared two T distributions fromsimulation and/or experiment using various similarity metrics, particularly compatibility dis-tance. Comparison between distributions from two consecutive time bins shows that the distri-butions from 3-4s and 4-5s time bins were qualitatively the same at every flux-tube, for both Teand Ti. The experimental data have additional uncertainties, therefore we broadened the spreadof the split-Gaussian distribution by adding 10% of µ to σ while maintaining the overall trendof the distribution. By comparing simulation distribution at 4-5s time interval with data fromshot #36266 or #36297, we found the two distributions are quantitatively the same. However,we also learned that the simulation Te profile at 7-8s looked different from the one at 4-5s. If weutilized compatibility distance as described to determine a quasi-steady state (e.g. at 5 s), thenwe might terminate the simulation too early and therefore we need to develop a more sensi-ble metric. While verification of and SA to MFW are work in progress, level of refinement andSobol SA are applied to a simple fusion model to show 3rd order PCE is sufficient for convergedresults and Te variance is most sensitive to 3 out of 5 varied inputs. This result can contributetoward dimension reduction of the model.References[1] O.O. Luk, O. Hoenen, A. Bottino, B.D. Scott and D.P. Coster, Comput. Phys. Commun. 239, 126-133 (2019)[2] D. Groen et al. Int. Conf. Comput. Sci. Springer, Cham, 479-492 (2019)[3] O. O. Luk et al. Phil. Trans. R. Soc. A. 379.2197, 20200074 (2021)[4] R. A. Richardson et al. J. Open Res. Softw. 8:1 (2020)[5] L. E. Veen and A. G. Hoekstra. Int. Conf. Comput. Sci. Springer, Cham, 425-438 (2020)[6] D. Suleimenova et al. J. Comput. Sci. 101402 (2021)47th EPS Conference on Plasma Physics P2.1075

Performing Validation, Verification, and Sensitivity Analysis on Multiscale Fusion Plasma Simulations with the VECMA Toolkit

http://hdl.handle.net/21.11116/0000-0009-A9D3-A

Performing Validation, Verification, and Sensitivity Analysis on Multiscale Fusion Plasma Simulations with the VECMA Toolkit

Abstract

Similar works

Full text

Available Versions

MPG.PuRe