Alef, Manfred

English

KITopen

Steinbuch Centre for Computing (SCC)www.kit.eduKIT  –  The Research University in the Helmholtz Association First Suggestions for a WLCG Fast BenchmarkManfred AlefWLCG GDB Amsterdam 2016-03-07Steinbuch Centre for Computing2 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkUse Cases and RequirementsUse cases:➔ Estimate the performance of the provided job slots in case of anonymous hardware (e.g. in commercial clouds)➔ Check the announced performance of the job slots (in batch farm)or a cloud VMSteinbuch Centre for Computing3 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkUse Cases and RequirementsRequirements:➔ As fast as possible● ~1 min runtime?➔ As exact as possible● Low spread of results● Be aware: the shorter the runtime of a fast benchmarkthe higher the probable inaccuracies What are the neighbors doing? Regular reassessments to iterate➔ Freeware (e.g. GPL), no licensing costsSteinbuch Centre for Computing4 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates5 fast benchmarks related to WLCG use casesAnalysis:➔ ~20.000 single-core batch jobs at GridKa measuring● The 1min system load at job start time● HS06 (from MJF)● Results of 4 fast benchmark candidates(+  ~2.500 jobs running candidate #5)➔ Time period: 2 weeks● GridKa farm utilization level varying between 50 and 100%Steinbuch Centre for Computing5 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "A"Dirac fast benchmark➔ Used by:● LHCb● Belle II, ...➔ Python script● Compiler flags are hidden to the user➔ Fast● Runtime: 0:30 ... 2:30 minutes➔ Correlation with job performance and with HS06:● Well scaling with LHCb workload [1,2]● Mismatch with HS06 of up to ~40% [3,4]Steinbuch Centre for Computing6 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "A"Dirac fast benchmarkSandy BridgeAMDHaswellSteinbuch Centre for Computing7 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "B"Atlas KV (KitValidation) benchmark➔ Used by:● Atlas➔ Framework for benchmarking, various benchmarks can plug-in● Wrapper script "kv-script.sh" downloaded from [5] Somewhat modified to run in batch environment instead of VM● Tested with default workload (Geant4: 100 Single Muon Events)● Compiler flags are hidden to the userSteinbuch Centre for Computing8 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "B"Atlas KV (KitValidation) benchmark➔ Very slow● Runtime: 3:45 ... >10:00 minutes(average: 6 ... 7 minutes)● Includes at least 3 minutes for pre- and post processing andregistering with external DB➔ Correlation with job performance and with HS06:● Scaling well with Atlas simulation jobs [6]● Probable mismatch with HS06 of up to 40%?● KV score is in units of seconds/event, in this talk:converting to inverse results ("KV speed")Steinbuch Centre for Computing9 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "B"Atlas KV (KitValidation) benchmarkSandy BridgeAMDHaswellSteinbuch Centre for Computing10 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "C"ROOT stress test➔ Used by:● Alice➔ By default using pre-compiled binary➔ Very fast● Runtime: 0:30 minutes➔ Correlation with job performance and with HS06:● Scaling linearly with Alice simulation efficiency [7]● Better scaling with HS06 than Dirac and KV(AMD’s about 30% faster than Intel’s if WN busy)Steinbuch Centre for Computing11 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidate "C"ROOT stress testSandy BridgeAMDHaswellSteinbuch Centre for Computing12 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates "D+E"Whetstone, Dhrystone➔ Used by:● Alice, Atlas, ...● HTCondor (condor_kflops, condor_mips), Boinc, ...➔ I have used the unmodified makefile (from UnixBench package [8])➔ Very fast● Runtime: 2:00 ... 3.00 minutes per benchmarkfor 10 iterations (default in UnixBench package) Can run less than 10 iterations➔ Very small memory footprint (less than L3 cache size)Steinbuch Centre for Computing13 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates "D+E"Whetstone➔ Correlation with job performance and with HS06:● Scaling very well with HS06 [3,4]● Probable mismatch with WLCG application performance➔ Not only the dispersion of the HS06-normalized results,but also their dependency on the system load, are very low● Most suitable candidate to estimate HS06 in cloud VM’s?Steinbuch Centre for Computing14 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates "D+E"WhetstoneSandy BridgeAMDHaswellSteinbuch Centre for Computing15 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates "D+E"Dhrystone➔ Benchmark results very similar to ROOTMARKSSteinbuch Centre for Computing16 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkBenchmark Candidates "D+E"DhrystoneSandy BridgeAMDHaswellSteinbuch Centre for Computing17 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkConclusions, Next StepsSuggesting 5 fast benchmark candidates (related to WLCG business) ➔ Quite good correlation of Whetstone with HS06,suitable candidate for fast HS06 estimate?➔ Dirac fast benchmark andKV Single Muon Simulationare scaling well with LHCband with Atlas simulationjobs● Good correlation ofboth fast benchmarks➔ ROOT stress test doesn’t correlate well with (most of) the other fast benchmarks, however it’s linear scaling with Alice jobsSteinbuch Centre for Computing18 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkConclusions, Next StepsPerformance measurements shown in this talk are the resultsof more than ~20.000 single-core batch jobs at GridKa,at global farm utilization level of around 50 ... 100%Cross-checks with other sites, with multi-core jobs, and alsoin relevant cloud environments, still to be done➔ Will relaunch the HEPiX Benchmarking Working Group soonSteinbuch Centre for Computing19 2016-03-07 Manfred Alef: First Suggestions for a WLCG Fast BenchmarkReferences[1] https://indico.cern.ch/event/319751/session/0/contribution/6/attachments/1153280/1656518/150909-MJFandBenchmarking-LHCb.pdf[2] https://indico.cern.ch/event/319754/session/0/contribution/8/attachments/1202029/1749779/151209-MJFUpdate-LHCb.pdf[3] https://indico.cern.ch/event/319754/session/0/contribution/9/attachments/1202373/1750492/Results_of_HS06_Scaling_Studies_at_KIT_2015-12-09.pdf{4] https://indico.cern.ch/event/433164/session/2/contribution/9/attachments/1220374/1783838/Results_of_HS06_Scaling_Studies_at_GridKa_2016-02-01.pdf[5] https://sdccloud.web.cern.ch/sdccloud/KV/DO-29401/index.html[6] https://indico.cern.ch/event/319751/session/0/contribution/8/attachments/1151865/1653919/gdb-20150910.pdf[7] https://indico.cern.ch/event/319751/session/0/contribution/9/attachments/1151876/1654130/ALICE_benchmarks.pdf[8] https://github.com/cloudharmony/unixbench

First suggestions for a WLCG fast benchmark

https://publikationen.bibliothek.kit.edu/1000053937/3831059

First suggestions for a WLCG fast benchmark

Abstract

Similar works

Full text

Available Versions

KITopen