Alef, Manfred

KITopen

KIT –  The Research University in the Helmholtz Association STEINBUCH CENTRE FOR COMPUTING  (SCC)www.kit.eduCPU Benchmarking at GridKa(Update April 2016, HEPiX, Zeuthen)Manfred AlefSteinbuch Centre of Computing2 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksSteinbuch Centre of Computing3 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksWhat’s the execution time of a simple arithmetic expression like this:Very easy exercise, isn’t it?a := b + cSteinbuch Centre of Computing4 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksWhat’s the execution time of a simple arithmetic expression like this:Very easy exercise, isn’t it?Load "b" and "c" into registersAdd "b" and "c"Store the result "a"a := b + cSteinbuch Centre of Computing5 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksWhat’s the execution time of a simple arithmetic expression like this:Very easy exercise, isn’t it?Load "b" and "c" into registersWhere are the most recent copies of "b" and "c"?Cache (L1, L2, ...), connected to which processor?Memory (RAM, swap)...Add "b" and "c"Store the result "a"a := b + cSteinbuch Centre of Computing6 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksWhat’s the execution time of a simple arithmetic expression like this:Are we talking about a single instruction, or about a loop?Some modern processors are coming with vector engines, e.g. AVX (2 operands) or AVX2 (3 operands)Execution time per vector element depends on vector length!Requirements:Compiler must support all relevant hardware featuresFeatures are enabled by compiler flagsBe aware that an application A1 will possibly run faster than A2 on hardware platform H1 while A2 runs faster on H2: ...a[0:n] := b[0:n] + c[0:n]a := b + cSteinbuch Centre of Computing7 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General Remarks     Intel NehalemIntel Sandy BridgeIntel HaswellAMD Magny-CoursAMD Abu Dhabi00,20,40,60,811,21,41,61,8Individual Benchmark Scores (per HS06)444 (FP) 447 (FP) 450 (FP) 453 (FP) 471 (INT) 473 (INT) 483 (INT) Individual Benchmark Scores (Scal d with HS06)Steinbuch Centre of Computing8 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksWhat’s the execution time of a simple arithmetic expression like this:Which other tasks are running at the same time?Fabric monitoringSystem updates...Concurrent VMs sharing the same physical hosta := b + cSteinbuch Centre of Computing9 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)General RemarksA benchmark can never scale exactly with each individual application (nor class of applications) on all relevant hardware typesBenchmarks should scale with a typical application, or with a representative mix of applicationsHS06 shall scale with the typical WLCG job mix"Good enough" (10% initial objective)Proved good correlation between HS06 scores and representative WLCG applications on various hardware platformsRemember that HS06 themselves is a mix of benchmarks(3 INT, 4 FP)Steinbuch Centre of Computing10 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Issues with the Default Benchmark (HS06)Steinbuch Centre of Computing11 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Issues with the Default Benchmark (HS06)Philippe Charpentier (LHCb)  –  see GDB 2015-09-09 + 2015-12-09:Performance of LHCb jobs (comparing with HS06 score of the provided batch slot) about 45% faster on WNs with Haswell processors, and around 30% on WNs with AMD chips,than on hosts with Sandy Bridge processorsDirac (LHCb) fast benchmark results looking very similarPerformance of Atlas jobs ('nevent/cpuconsumptiontime' from bigpanda store, compared with HS06):Similar to the LHCb results for Atlas simulation jobs  :-(Linear scaling for all other types of Atlas jobs  :-)Alice, CMS?Steinbuch Centre of Computing12 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Issues with the Default Benchmark (HS06)Probable reasons:Some jobs are possibly enabling platform dependent hardware features, e.g. vector engines (AVX2)HS06 is based on the default compiler (gcc-4.4.x on SL6), and on a mandatory set of flags ('–O2 –pthread –fPIC –m32')On Haswell box: 444.namd runs 20% faster when compiled with latest compiler gcc-5.3.x and with more aggressive flags'–O3 –m64 –march=haswell'Are WLCG experiments using their own, more recent compiler implementations?Hardware features are possibly enabled by default in SL6 Python packages (boosting Dirac fast benchmark)Bonus performance for user jobs, no degradationSteinbuch Centre of Computing13 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksSteinbuch Centre of Computing14 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksHS06 is a tool suitable to describe installed capacities, procurements, accounting, fair-share batch scheduling, aso, and must scale well with the typical job mix running at WLCG sitesHowever, HS06 is inapplicable in several scenarios, for instance users cannot run it to estimate the performance of the provided batch slot or VM:RuntimeLicense issuesDemand for a fast benchmark to estimate the performanceRuntime ≈ 1 minuteProbably less precise than full benchmark, but "good enough"Free licenseSteinbuch Centre of Computing15 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksSuggesting 5 candidates:Dirac (LHCb) fast benchmarkLHCb, Belle II, ...Atlas KV / Geant4 single muon eventsAtlasROOT stress testAliceWhetstone, DhrystoneHTCondor, Boinc, ...Steinbuch Centre of Computing16 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksFirst investigations:Measurements in GridKa batch farm≈ 20,000 single-core jobsSystem load at job start timeHS06 of the provided slot (from MJF)Running several fast benchmarksFarm utilization level varying between 50% and 100%Results ...Steinbuch Centre of Computing17 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksDirac fast benchmark                                            0:30 ... 2:30 minutesSandy BridgeAMDHaswellSteinbuch Centre of Computing18 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksAtlas KV (KitValidation) benchmark                       6 ... 7 (–3) minutesSandy BridgeAMDHaswellKV Speed (Single Muon Event Generation) / System LoadSteinbuch Centre of Computing19 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksROOT stress test                                                               0:30 minutesSandy BridgeAMDHaswellSteinbuch Centre of Computing20 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksWhetstone (UnixBench)                            2 ... 3 minutes (10 iterations)Sandy BridgeAMDHaswellSteinbuch Centre of Computing21 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Fast BenchmarksDhrystone (UnixBench)                             2 ... 3 minutes (10 iterations)Sandy BridgeAMDHaswellSteinbuch Centre of Computing22 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)HEPiX Benchmarking Working GroupSteinbuch Centre of Computing23 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)HEPiX Benchmarking Working GroupFirst informal meeting tomorrowWill start with deeper look into scaling and accuracies of fast benchmarksPerformance estimation of anonymous hardware,e.g. commercial cloudsScaling with typical WLCG applications running on various hardware (VM) modelsSpread of results, e.g. depending on system loadWhat are the neighbours doing?Multi-core benchmarkingSteinbuch Centre of Computing24 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)ConclusionsSteinbuch Centre of Computing25 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)ConclusionsThe official WLCG benchmark HS06 doesn’t scale with some of the most important applications on some hardware modelsMismatch of up to 45% (underestimation)The intended use cases of HS06 are installed capacities(accounting, procurements, fair-share batch scheduling, aso.)HS06 must scale well (≈ 10%) with typical job mix at WLCG sitesPerformance prediction of individual jobs is not an intended use case!Complementary (fast) benchmark(s) useful(Repeating) accuracy less importantRelaunch of the HEPiX Benchmarking Working GroupStarting with fast benchmarksSteinbuch Centre of Computing26 2016-04-20 Manfred Alef: CPU Benchmarking at GridKa (Update April 2016)Questions, Comments?

CPU benchmarking at GridKa (Update April 2016)

https://publikationen.bibliothek.kit.edu/1000054129/3835127

CPU benchmarking at GridKa (Update April 2016)

Abstract

Similar works

Full text

Available Versions

KITopen