National audienceIn recent years, much research has been conducted in the area of energy efficiency in distributed systems. To analyze, understand and improve their behaviour, simulators provide useful tools, to achieve energy-aware simulation like DVFS (Dynamic Voltage and Frequency Scaling). This paper presents current work on Grid'5000 to deploy a specific distributed electromagnetic application called TLM (Transmission Line Matrix), using DVFS and power measurements. The aim is to launch different set of experiments using different DVFS configurations, and then compare simulations and real experiments results

Alexandru, Mihai

Da Costa, Georges

Guérout, Tom

Monteil, Thierry

da Costa, Georges

HAL-INSA Toulouse

Grid'5000 energy-aware experiments with DVFS

Scientific Publications of the University of Toulouse II Le Mirail

Grid’5000 energy-aware experiments with DVFSTom Gue´rout, Georges Da Costa, Thierry Monteil, Mihai AlexandruTo cite this version:Tom Gue´rout, Georges Da Costa, Thierry Monteil, Mihai Alexandru. Grid’5000 energy-awareexperiments with DVFS. grid’5000 school, Dec 2012, Nantes, France. 7p., 2012. <hal-01228321>HAL Id: hal-01228321https://hal.archives-ouvertes.fr/hal-01228321Submitted on 12 Nov 2015HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.Grid’5000 energy-aware experiments with DVFSTom Gue´rout123, Georges Da Costa123, Thierry Monteil13, and MihaiAlexandru131 CNRS; LAAS; 7 avenue du Colonel Roche, F-31077 Toulouse, France,2 IRIT; 118 route de Narbonne, 31000 Toulouse, France,3 Universite´ de Toulouse; UPS, INSA, INP, ISAE; LAAS; F-31077 Toulouse, Francetguerout@laas.frAbstract. In recent years, much research has been conducted in thearea of energy efficiency in distributed systems. To analyze, understandand improve their behaviour, simulators provide useful tools, to achieveenergy-aware simulation like DVFS (Dynamic Voltage and FrequencyScaling). This paper presents current work on Grid’5000 to deploy aspecific distributed electromagnetic application called TLM (Transmis-sion Line Matrix), using DVFS and power measurements. The aim is tolaunch different set of experiments using different DVFS configurations,and then compare simulations and real experiments results.Keywords: power consumption, dvfs, simulations1 IntroductionNowadays, the increasing use of datacenters makes the analysis of energy con-sumption more and more important. Some metrics to evaluate their efficiencyare known (PUE, ERE) [1] and much research are ongoing to find new ways toreduce the energy. This leads to improvement and/or creation of new algorithmsthat require extensive testing phases to validate their effectiveness.To compare real energy-aware experiments, some simulators allow reducingthe consumption of machines/cpu, while others give the possibility to measurethe total energy consumption of a simulation, but it seems that none of themprovide all tools required to achieve a complete energy-aware simulator. For theseexperiments, the simulator CloudSim has been improved with DVFS support tobe able to run simulations with similar behaviour as Grid’5000 experiments.After the DVFS implementation into CloudSim is validated with a simpletest application, the next step is to compare Grid’5000 experiments of a realdistributed application, the TLM, using DVFS and power measurements withCloudSim simulation results.This paper details specifically a TLM experiment. It is organized as follows.First section 2.1 explains how to find the optimal frequency. Section 2.2 showshow experiments have been conducted in term of energy and how to configureDVFS. Results of these experiments are analyzed in Section 2.3, and the cur-rent work on hard disk throughput is explained in section 3. Finally, section 4concludes the paper.22 Deployment of TLM Electromagnetic applicationTLM implements a numerical method for the electromagnetic simulation whichfills the environment of the electromagnetic field propagation with a network oftransmission lines. This propagating field model inside a given medium becomespossible thanks to the equivalence that exists between the electric and magneticfields and voltages and currents in a transmission line network. A detailed pre-sentation of TLM method, also in a three-dimensional approach, can be foundin [2].The parallel resolution process of this method on a grid is based on a divisionof the structure along the three axes. Volumes obtained are assimilated to tasksexecuted on each machine, CPU, or CPU core, sending data over MPI library(Message Passing Interface).The aim of these experiments is to compare the power consumption usingdifferent DVFS configurations (i.e different frequencies). After referencing powerconsumption of Reims nodes at 0% and 100% of use, the next step is to determinethe theoretical optimal fixed frequency.2.1 CPLEX SolverIBM CPLEX solver was used to compute the optimal fixed frequency, depend-ing on TLM input parameters (size of the structure and number of iterations),network bandwidth and latency and power values of machines. The problem hasbeen written in JAVA and described as follows:H : Current node usedTnet and Tcpu : Network and CPU timeF = {f1, f2, ..., fn−1, fn} : Available frequencies on node H , with fn−1 < fn.lHcpu(fi) et dHcpu(fi) : Power of node H at 0% and 100% of CPU load.zfi : Binary variable, equal to 1 if fi is used, 0 otherwise.The prediction model of CPU time and network TLM presented in [3] wasused:Tcpu = C1+nlnxnynzC2 , Tnet =(L+nxnyD)∗4nl , with nx, ny,nz dimensionsof TLM environment, nl the number of iterations, C1 et C2 constants estimatedby past experiments, L et D are latency and network bandwidth.The expression of the energy E to be minimized is:E = Tnet ∗[lHcpu(f1) ∗ zfa + ...+ lHcpu(fn) ∗ zfn]+(Tcpu ∗ fn)[dHcpu(f1)∗zfaf1+ ...+dHcpu(fn)∗zfnfn]with constraint :∑ni=1 zfi = 1, which allows to use only one frequency.The energy equation E takes into account available host frequencies andpower consumption values. Reims Grid’5000 values are in Table 1.With these available frequencies and power consumption values at Idle andFull state on all 24 cores of a node, CPLEX solver is able to find the optimal3Table 1. Frequencies available and power consumption values (at idle and full state)for each frequency on Grid’5000 Reims siteReims site characteristicsAvailable0.8 1.0 1.2 1.5 1.7Frequencies (GHz)Power Idle 140 146 153 159 167Consumption (W) Full 228 238 249 260 272frequency that minimize E for different ratio valuesTcpuTnet. For example, oneinstance of this problem with the ratioTcpuTnet= 1 looks like :Minimizeobj: 52.0406573169004 z_a + 45.8853564547206 z_b + 42.1450984558719 z_c+ 37.8042459957691 z_d + 36.5833333333333 z_eSubject Toc1: z_a + z_b + z_c + z_d + z_e = 1Bounds0 <= z_a <= 10 <= z_b <= 10 <= z_c <= 10 <= z_d <= 10 <= z_e <= 1Binariesz_a z_b z_c z_d z_eEndOnce all energy consumption values computed (for each frequencies) andthe optimal frequencies for each ratioTcpuTnetfound by the solver, it’s interesting tomerge these two results in one figure (shown in figure 1). The TLM configurationused for these experiments give a ratio of about 0.16, so the theoretical optimalfrequency that will be use is 1.5GHz.2.2 Experiments on Grid’5000These experiments were conducted on the Grid’5000 Reims site, whose config-uration is: HP ProLiant DL165 G7 CPU: AMD Opteron 6164 HE 1.7GHz, 12MB L3 cache, 44 nodes with 2 CPUs to 12 cores per CPU (1056 cores). TLMwas deployed on 2 nodes used at 100 %, indeed using 48 cores, with one taskTLM on each core.Input parameters are: TLM parameters : nx=172 ny=90, nz=12 and nl=26000.Throughput and latency measured (with a simple MPI application) in Reims are: D=700Mbit/s and L=4*10−5s.Deployment of TLM on Grid’5000 Reims site has been done in a Kadeployenvironment. Once the image has been deployed on all nodes, the first step is toconfigure the DVFS. In this experiment, 3 modes has been used :– Maximum Frequency (1.7GHz) in Performance mode.4 0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8 0.4 0.5 0.6 0.7 0.8 0.9 1 0 5 10 15 20 25 30 35 40 45 50Energy (Wh)Optimal Frequency with CPLEXEnergy consumptionOptimal Energy ValueTime Ratio CPU/(Net+Disk)Frequency (%*FMax)Energy (Wh)Fig. 1. Optimal frequencies (in green) for each ratio values– Optimal Frequency (1.5GHz) determined by CPLEX solver in UserSpacemode.– OnDemand mode, which allow the kernel to dynamically change the fre-quency of the CPU regarding its current load.Then, another bash script to manage the power consumption is launched. First,it has to wait for the start of the experiment and then save power consumptionof all deployed nodes into files. Each second the script runs the command lineto get the power consumption. At the end of the experiment, all output files aresent to the frontend.2.3 ResultsTable 2 show all detailed experiments results in term of duration and energyconsumption values on each node.Results analysis Simulator results show the expected logic: theoretical opti-mum frequency given by the solver obtains a power consumption value lowerthan the maximum frequency, despite a longer execution time and OnDemandmode is more efficient with a gain of -10.5% relative to the optimum frequency.Network latency and throughput are hidden in the simulator, while real experi-ments are strongly influenced by these changes. In analysing the results of twodifferent modes that have been executed on the same node, or two experimentsrunning at same frequency on both same nodes, we can note that this is only thecommunication time between the tasks which has varied. This variation disruptsthe overall result, while the mode change only DVFS impacts the computationaltime and does not affect the time spent in communication. The standard de-viation value indicate high dispersion in all measures, due to to the variationof network performance on the environment used. For calculating energy PDUs5Table 2. Detailed results of experiment on Grid’5000 Reims siteDVFS modeDuration (s) Energy (Wh)NodesTotal % Error Node 1 Node 2 Total % ErrorPerformance (1.7GHz)7788 -0.15 391 372 763 -6.4 25-267953 1.96 332 424 756 -7.2 1-108529 9.34 398 427 825 1.22 5-78627 10.6 457 443 900 10.4 27-298681 11.3 453 465 918 12.6 25-26Optimal (1.5GHz)8030 -0.37 393 400 793 -2.57 27-288071 0.14 400 416 816 0.24 25-268686 7.76 433 439 872 7.12 25-26OnDemand8084 3.64 401 394 795 7.43 2-208088 3.69 384 405 789 6.62 2-218170 4.74 431 407 838 13.24 2-208248 5.74 400 430 830 12.16 5-78324 6.71 406 440 846 14.32 3-288401 7.71 421 440 861 16.35 3-288603 10.29 439 475 914 23.51 25-26available on the website of Reims were used once per second, but their powermeasures are updated only each 3 seconds which also affects the accuracy of thetotal energy consumed value.Table 3. Comparison between Grid’5000 experiments and CloudSim (bolds values)DVFS ModeDuration (s) Energy (Wh)Average Deviation % Error Average Deviation % ErrorPerformance (1.7GHz)8320 8338315 414 -0.06 832 75 -0.12Optimal (1.5GHz)8580 8318262 367 -3.7 832 30 0.12OnDemand8320 7438320 367 0 827 40 11.313 Next step of Grid’5000 experimentsThe TLM application need to write a lot of informations into several files. Toavoid overloading the RAM, the application can write a part of these data duringeach iteration. The first step is to estimate the rate of hard drive in Reims todetermined the time needed to write these data at each iterations. To do thesemeasures, a C language application has been implemented. This applicationwrites files from 20Mo to 180Mo , ten times for each size, and compute theaverage and the standard deviation to estimate the rate. Table 4 show results ofthe first measures :6Table 4. Hard Disk throughput measures on Reims siteFile size (Mo) Average (s) Standard Deviation (s) Throughput (Mo/s)20 0.038072 0.0031 525.3240 0.038072 0.0030 517.3760 0.115149 0.0030 521.0680 0.154235 0.0044 518.69100 0.193702 0.0038 516.26120 0.239641 0.0069 500.75140 0.283617 0.0091 493.6160 0.341733 0.0877 468.20180 0.379805 0.0484 473.93With these values, the aim is to determine how much information can bewritten in each iteration of TLM, without too much slow down the application.To do that, others experiments will be done, CPU time and Network time willbe estimate with different configuration of TLM input parameters, and the filesize to write will be determine to have an efficient CPUNetwork+I/O time ratio.4 ConclusionThese experiments highlights the difficulty to obtain reliable power consumptionvalues on a real platform, here due to the large standard deviation betweenexperiments. One reason is that power measurements are not enough precisebut also because the communication time varies a lot depending on experimentsand resources used. To show the efficiency of DVFS, next TLM experiments willhave a shorter communication time and use also I/O more constant.References1. Patterson, M., Tschudi, B., Vangeet, O., Cooley, J., Azevedo, D.: ERE: A metric formeasuring the benefit of reuse energy from a data center. Technical Report WhitePaper 29, The Green Grid (2010)2. Hoeffer, W.: The transmission-line matrix method–theory and applications. Mi-crowave Theory and Techniques, IEEE Transactions on 33(10) (oct 1985) 882–8933. Alexandru, M., Monteil, T., Lorenz, P., Coccetti, F., Aubert, H.: Efficient large elec-tromagnetic problem solving by hybrid tlm and modal approach on grid computing.In: International Microwave Symposium, Montral, Canada, 17-22 june 2012. (2012)

https://hal.archives-ouvertes.fr/hal-01228321

Grid'5000 energy-aware experiments with DVFS

Abstract

Similar works

Full text

Available Versions

HAL-INSA Toulouse

Scientific Publications of the University of Toulouse II Le Mirail