This paper presents an approach to a comparative evaluation of the predictive ability of spallation reaction models based on widely used, well-proven multiple-criteria decision analysis methods (MAVT/MAUT, AHP, TOPSIS, PROMETHEE) and the results of such a comparison for 17 spallation reaction models in the presence of the interaction of high-energy protons with natPb

Andrianov, Andrey

Andrianova, Olga

Konobeev, Alexandr

Korovin, Yury

Kuptsov, Ilya

EPJ Web of Conferences

KITopen

EPJ Web of Conferences 146, 12007 (2017) DOI: 10.1051/epjconf/201714612007ND2016Multi-criteria comparative evaluation of spallationreaction modelsAndrey Andrianov1, Olga Andrianova2, Alexandr Konobeev3, Yury Korovin4,a, and Ilya Kuptsov11 National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia2 Joint Stock Company “State Scientific Centre of the Russian Federation – Institute for Physics and Power Engineering named afterA.I. Leypunsky”, Obninsk, Russia3 Institute for Neutron Physics and Reactor Technology, Karlsruhe Institute of Technology (KIT), Germany4 Obninsk Institute for Nuclear Power Engineering of National Research Nuclear University MEPhI, Obninsk, RussiaAbstract. This paper presents an approach to a comparative evaluation of the predictive ability ofspallation reaction models based on widely used, well-proven multiple-criteria decision analysis methods(MAVT/MAUT, AHP, TOPSIS, PROMETHEE) and the results of such a comparison for 17 spallation reactionmodels in the presence of the interaction of high-energy protons with natPb.1. IntroductionIn the last decade, high energy nuclear reactions haveattracted increasing interest due to scientific problems andnumerous applications, such as: creation of high-energyneutron sources, production of medical radioisotopes,radiation protection of space vehicles and accelerators.Solving these problems requires a large amount of nucleardata for a wide range of nuclides and energies up to severaldozens of GeV. To obtain all data experimentally is notpossible; therefore, it is necessary to develop analyticalmethods the accuracy of which should be verified bymeasurements performed under certain conditions [1–5].There is a growing number of models and programsdesigned to simulate nuclear reactions in different energyranges and mass numbers as well as criteria and algorithmsto verify the adequacy of simulated nuclear reactions tofull-scale experimental data [6–8]. It should be notedthat currently there are no universally accepted theoreticalconcepts and models that could satisfactorily explain theentire spectrum of the considered nuclear reactions. Anevaluation of the predictive ability of calculation tools isbased on the goodness-of-fit test. Due to lack of consensusamong experts in the subject area on goodness measuresand assessment procedures, the evaluation results may varysignificantly.This paper presents an alternative approach toevaluating the predictive ability of models based on thewidely used, well-proven methods for discrete decisionanalysis (MAVT/MAUT, AHP, TOPSIS, PROMETHEE)and the results of such evaluations for the known spal-lation reaction models (Bertini/Dresner, Bertini/ABLA,ISABEL/Dresner, ISABEL/ABLA, INCL4/Dresner,INCL4/ABLA, CEM2k, CASCADE, CASCADE/ASF,CASCADEX-1.2 [9]) in the presence of the interaction ofhigh-energy protons with natPb.a e-mail: korovinyu@mail.ru2. Multiple-criteria decision analysisMultiple-criteria decision analysis (MCDA) methods areaimed at supporting decision makers who have to dealwith numerous and conflicting assessments and intend tohighlight conflicts and find compromises in the decisionmaking process [10,11]. The MCDA problems consist ofa finite number of alternatives explicitly known at thebeginning of the decision support process. Each alternativeis characterized by its performance in multiple criteria. Theproblem may be defined as finding the best alternative fora decision maker, or selecting a set of acceptable trade-offalternatives.The MCDA methods provide an opportunity for ananalysis of the predefined set of alternatives. MCDAis applicable to the following problem: given a set ofalternatives and criteria for their assessment, it can beassumed that each alternative has been evaluated byeach criterion either by experts’ judgments or objectivecalculations. Then, it is necessary to derive a rule from theexperts’ preferences which will make it possible to rankthe alternatives according to their values and identify thebest among them.A large number of MCDA methods have beendeveloped to deal with different kinds of problems.The evaluations presented in this study were madeusing the following well-known and widely used MCDAmethods: MAVT (Multi-attribute Value Theory), MAUT(Multi-attribute Utility Theory), TOPSIS (Technique forOrder Preference by Similarity to the Ideal Solution),PROMETHEE (Preference Ranking Organization Methodfor Enrichment Evaluations), AHP (Analytic HierarchyProcess).3. Deviation factorsTo apply these methods to evaluations of the predictiveability of spallation reaction models in the presence ofthe interaction of high-energy protons with nuclei, thec© The Authors, published by EDP Sciences. This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0(http://creativecommons.org/licenses/by/4.0/).EPJ Web of Conferences 146, 12007 (2017) DOI: 10.1051/epjconf/201714612007ND2016Table 1. Deviation factors.Evaluation of the integrated closeness to the experiment provided that the data can be very different.The index of a power reflects the acceptable compensation degree for small values of some parameters by large values of others. The higher the index, the greater the possible compensation degree.Evaluation of the relative integral closeness to the experiment.following reactions for natPb were selected. The selectionof this set of nuclei is due to the fact that there is a largeset of experimental data for natPb because it is consideredas the basic one for a number of innovative reactor facilityprojects.The experimental values were taken from the EXFORdatabase as well as databases used in the IAEA“Benchmark of Spallation Models” project [12–14].Table 1 shows the deviation factors used in thepresent study. Table 2 contains the deviation factor valuesfor the natPb(p,x) reaction. To evaluate the factors, 279experimental values were selected for natPb residual cross-sections at the projectile proton energy values in the rangeof 70–2600 MeV (see Fig. 1).The problem is to rank the models in accordance withthe values of the totality of the deviation factors. Thisrequires aggregation which is ensured by the use of theMCDA methods.The studies properly organized on the basis ofthe MCDA paradigm represent a process as not onlytechnically operating with a set of mathematical methodsand various analytical tools but also leading to acomprehensive understanding of the problem and itselaboration. MCDA does not provide a ‘right solution’;in this regard, it would be appropriate to speak abouta compromise or a trade-off solution, paying specialattention to an analysis of the solution stability to thevarious methods used and their model parameters.4. Ranking of models using differentMCDA methodsThe approach applied in this study consists of severaldifferent MCDA methods which may facilitate a thoroughunderstanding, recognition and analysis of the problem,providing an additional sensitivity analysis of the obtainedranking results to the methods used that increase theconfidence level of the study.Application of a wide range of different methods mayhave a significant impact on subsequent decision makingand help a decision maker more thoroughly understandand analyze the problem in order to achieve consistencyin judgments and estimates. At the same time, it becomesTable 2. Deviation factors for the natPb(p,x) reaction.Models Deviation factorsH D R FCascade 4 6.17 0.69 0.91 5.14Cascade/ASF 4.62 0.49 0.91 2.57CASCADeX 1.2 5.82 0.71 0.46 10.98CEM02 4.84 0.51 1.05 2.44CEM03 5.21 0.56 1.06 2.46geant4/bertini 14.80 1.02 1.40 4.00geant4/binary 4.39 0.53 0.69 3.73incl45/Abla07 9.61 0.81 1.51 2.04incl45/gemini 20.26 1.28 2.04 2.48incl45/smm 9.57 0.87 1.27 3.67Bertini/Dresner 7.37 0.72 1.15 2.59Isabela/Abla07 13.13 1.08 1.77 2.29Isabel/Gemini 30.30 1.70 2.49 2.79Isabela/smm 10.04 0.92 1.35 4.04phits/jqmd 42.86 2.23 2.26 6.43phits4/jam 5.63 0.54 0.93 2.12phits/bertini 6.75 0.61 1.16 2.08necessary to examine the stability and robustness of theranking results towards different assumptions.Although the ranks of alternatives may vary fordifferent MCDA methods, an analysis of the problemby different methods can play a significant role in theinteractive process of understanding the problem andidentifying its main features and demonstrate that differentmethods may provide noncontradictory results.All the methods are implemented in their simplestform. The assumption of equivalence of all the deviationfactors is accepted as the basic option. Changes of thedeviation factors’ significance (weights) were made withina sensitivity analysis.Table 3 shows the models ranking results (ranks) usingvarious methods for equal weight option and their combin-ing into groups. As shown in Table 3, the use of variousdiscrete decision analysis methods (MAVT/MAUT, AHP,TOPSIS, PROMETHEE) to evaluate the predictive abilityof spallation reaction leads, despite some differences inmodel ranking, to well-coordinated and similar results.Despite the fact that the models ranking results aresomehow affected by the deviation factors’ weights, thereare stability regions where the ranking order is preservedin a wide range of weight value changes (Fig. 2). However,models of a certain group do not overrun its borders.To obviate the necessity of defining the values ofweighting factors for a multi-attribute model, a weightstochastic generation method has been implemented whichmakes it possible to rank models in the absence ofinformation about the significance of certain deviationfactors (weights values) as well as determine themodel preference probability, i.e., integral performancecharacteristic of a computational model.Figure 3 shows the model ranking results with dueregard to the uncertainty in the weight values in a box plotformat (95, 75, 50, 25, 5% percentiles are denoted; themodels are ranked by the average scores, i.e., the valuesof a multi-attribute value function). The models rankingresults based on this approach are consistent with theresults obtained from the classical deterministic methodsdescribed above.2EPJ Web of Conferences 146, 12007 (2017) DOI: 10.1051/epjconf/201714612007ND2016Figure 1. Excitation function for the natPb(p,207Bi) reaction, experiment values [14].Table 3. Ranking results for the natPb(p,x) reaction.RankMCDA methods#MAVT/AHP TOPSIS PROMETHEEMAUT1 CEM02 CEM02 phits4/jam CEM0212 phits4/jam phits4/jam CEM03 CEM033 Cascade/ASF CEM03 phits/bertini phits4/jam4 CEM03 Cascade/ASF Cascade/ASF Cascade/ASF5 phits/bertini phits/bertini CEM02 phits/bertini6 Bertini/Dresner Bertini/Dresner phits/jqmd Bertini/Dresner27 Cascade 4 Cascade 4 Isabela/smm Cascade 48 INCL45/abla07 INCL4/abla07 Cascade 4 INCL45/smm9 INCL45/smm Isabela/smm INCL45/abla Isabela/smm10 geant4/ binary geant4/binary geant4/ binary geant4/binary11 Isabela/smm INCL4/smm INCL45/gemini geant4/bertini12 geant4/bertini geant4/bertin Bertini/Dresner INCL4/abla0713 Isabela/Abla07 Isabela/Abla07 geant4/bert geant4/bertin314 INCL45/Gemini INCL45/gemini Isabel/Gemini INCL45/gemini15 CASCADeX1.2 CASCADeX1.2 INCL45/smm CASCADeX1.216 Isabel/gemini Isabel/gemini Isabela/Abla07 Isabel/gemini17 phits/jqmd phits/jqmd CASCADeX1.2 phits/jqmdFigure 2. Linear weight approach to weights sensitivity analysis.3EPJ Web of Conferences 146, 12007 (2017) DOI: 10.1051/epjconf/201714612007ND2016Figure 3. Ranking of models for calculating the natPb(p,x)reaction.The study demonstrates that taking into accountthe sensitivity analysis results, an additional alternativeanalysis using experts’ judgments and the whole set ofgraphical and attributive information, it is possible to selectthe best models.The best models can be considered those ofthe first group including: CEM02, CEM03, Phits/jam,Cascade/ASF, Phits/Bertini. The models Bertini/Dresner,Cascade-4, INCL4/ABLA, INCL4/SMM, geant4/binary,Isabela/SMM, geant4/Bertini may be referred to thesecond in attractiveness group. The models Isabela/Abla,INCL4/Gemini, CASCADeX-1.2, Isabel/Gemini, phits/jqmd, which are characterized with a greater uncer-tainty, may be united into the next in attractivenessgroup.A multi-criteria approach to a comparative evaluationof high-energy nuclear reaction models as well asevaluated nuclear data obtained by using these modelsmakes it possible to more finely differentiate variousmodels with due account for experts’ opinions, whichmakes an additional contribution to both the understandingof nuclear reaction mechanisms and preparation of areliable nuclear data set.5. ConclusionThe study has shown that if the MCDA methods areapplied to evaluating the predictive ability of spallationreaction models, despite some differences in modelranking, the results obtained by using different methodsturn out to give good fits. The study demonstrates thattaking into account the sensitivity analysis results, anadditional alternative analysis using experts’ judgmentsand the whole set of geographical and attributiveinformation, it becomes possible to select the best models.The study was partly performed under the support from theMinistry of Education and Science of Russia (unique identifierof applied scientific project 11.9655.2017/bch).References[1] V.V. Artisuk, Yu.A. Korovin, A.V. Ignatyuk,G.B. Pilnov, A.Yu. Stankovsky, Yu.E. Titarenko,S.G. Yavshits, Pramana-journal of physics, IndianAcademy of Science 68(2), 181–191 (2007)[2] http://nds121.iaea.org/alberto/mediawiki-1.6.10/index.php/Main Page[3] S. Leray, PSI Proceedings 09-01 ARIA, p. 89 (2009)[4] V. Artisyuk, Yu. Korovin, C. Broders, E. Gonzalez-Romero, A. Konobeyev, et al., Progress in nuclearenergy 50, 341–346 (2008)[5] A.A. Andrianov, Yu.A. Korovin, G.M. Pshakin,Journal of nuclear material management, XXXVII 1,48–54[6] A.Yu. Konobeev, Yu.A. Korovin, A.A. Natalenko,S.A. Osikin, G.B. Pilnov, A.Yu. Stankovsky,A.V. Tihonenko, U. Fisher, Izvestiya Vuzov,Yadernaya Energetika, N02, 8–15 (2007)[7] A.A. Natalenko, Yu.A. Korovin, A.Yu. Konobeyev,A.Yu. Stankovskiy, S.G. Mashnik, Los AlamosNational Laboratory Report.– 2010. – LA-UR-10-01397[8] Yu.A. Korovin et al., Journal of the Korean PhysicalSociety 59(2), 1080–1083 (2011)[9] A.A. Andrianov, A. Konobeev, Yu.A. Korovin,I.S. Kuptsov, A.Yu. Stankovsky, Izvestiya Vuzov,Yadernaya Energetika, N02 (2011)[10] V. Kuznetsov, G. Fesenko, A. Schwenk-Ferrero, A.Andrianov, I. Kuptsov, Energies 8, 3679–3719 (2015)[11] V. Kuznetsov, G. Fesenko, A. Andrianov,I. Kuptsov, Science and Technology of NuclearInstallations, Article ID 910162, 15 pages,doi: 10.1155/2014/910162 (2014)[12] M. Gloris, R. Michela, F. Sudbrock, U. Herpers,P. Malmborg, B. Holmqvist, Nuclear Instrumentsand Methods in Physics Research Section A:Accelerators, Spectrometers, Detectors and Associ-ated Equipment 463(3), 593–633 (2001)[13] I. Leya, R. Wieler, J.-C. David, S. Leray,L. Donadille, J. Cugnon, R. Michel, NuclearInstruments and Methods in Physics ResearchSection B: Beam Interactions with Materials andAtoms, 229(1), 1–23 (2005)[14] Yu.E. Titarenko, V.F. Batyaev, R.D. Mulambetov,V.M. Zhivun, V.S. Barashenkov, S.G. Mashnik,Yu.N. Shubin, A.V. Ignatyuk, Nuclear Instrumentsand Methods in Physics Research Section A: Ac-celerators, Spectrometers, Detectors and AssociatedEquipment 562(2), 801–805 (2006)4

Multi-criteria comparative evaluation of spallation reaction models

Andrey Andrianov

Olga Andrianova

Alexandr Konobeev

Yury Korovin

Ilya Kuptsov

Crossref

EDP Sciences OAI-PMH repository (1.2.0)

https://publikationen.bibliothek.kit.edu/1000075810/4425826

Multi-criteria comparative evaluation of spallation reaction models

Abstract

Similar works

Full text

Available Versions

KITopen

Crossref

EDP Sciences OAI-PMH repository (1.2.0)