Lucarelli, Giorgio

Mommessin, Clement

Trystram, Denis

INRIA a CCSD electronic archive server

HAL Id: hal-02459646https://hal.inria.fr/hal-02459646Submitted on 29 Jan 2020HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Scheduling at the EdgeClement Mommessin, Giorgio Lucarelli, Denis TrystramTo cite this version:Clement Mommessin, Giorgio Lucarelli, Denis Trystram. Scheduling at the Edge. 14th Scheduling forLarge Scale Systems Workshop, Jun 2019, Bordeaux, France. pp.1-28. ￿hal-02459646￿Scheduling at the EdgeClément MommessinGiorgio Lucarelli Denis TrystramUniv. Grenoble Alpes, CNRS, INRIA, LIG, F-38000 Grenoble, FranceJune 27th, 2019Clément Mommessin Scheduling at the Edge June 27th, 2019 1 / 23Credits: https://www.gironde.fr/actualites/residence-florestine-innovation-technologique-et-socialeClément Mommessin Scheduling at the Edge June 27th, 2019 2 / 23From Heater to Computing Resources“A disruptive solution to turn IT waste heat into a viable heatingsolution for buildings.”The Qarnot platform:∼1,000 distributed QRads embedding∼3,000 diskless computing units(QMobos)∼20 local servers (QBoxes) with disks1 global server (QNode) with acentralized storage server (CEPH)Credits: https://www.qarnot.comClément Mommessin Scheduling at the Edge June 27th, 2019 3 / 23Qarnot PlatformInternetQNodeQBoxQRadCEPHClément Mommessin Scheduling at the Edge June 27th, 2019 4 / 23Qarnot PlatformInternetQNodeQBox IoT tasksQRadCEPHCloud tasksClément Mommessin Scheduling at the Edge June 27th, 2019 4 / 23Qarnot PlatformInternetQNodeQBox IoT tasksQRadCloud tasksCEPHHeatingrequestsClément Mommessin Scheduling at the Edge June 27th, 2019 4 / 23Computing RequestsCloud tasks:Submitted to the QNodeHave data-set dependencies in the centralized storage (CEPH)Have different priorities (low or high)IoT tasks:Submitted to a QBoxHave data-set dependencies in the QBox diskHave different priorities (low, high or very high)Should be executed locallyTasks (= groups of sequential instances) are submitted on-line.Clément Mommessin Scheduling at the Edge June 27th, 2019 5 / 23Platform DynamicityResources appear and disappear over time: the inhabitants decide!Available resources when heating is required (QRad is On)Unavailable resources when ambient air is too warm (QRad is Off)→ Also depends on the task priorityNetwork uncertainties:Link failuresCongestion/contentionClément Mommessin Scheduling at the Edge June 27th, 2019 6 / 23Two-Level Scheduling ProblemThe only computing power is in the QRadMake global decisions at QNode-level:Decide where to dispatch (groups of) instancesEnsure global load-balancingMake local decisions at QBox-level:Schedule instances on QRadsRegulate room temperature (via DVFS)Ensure heating needs are satisfiedDispatch instancesSchedule instancesQNodeQBoxQRadClément Mommessin Scheduling at the Edge June 27th, 2019 7 / 23Two-Level Scheduling ProblemThe only computing power is in the QRadMake global decisions at QNode-level:Decide where to dispatch (groups of) instancesEnsure global load-balancingMake local decisions at QBox-level:Schedule instances on QRadsRegulate room temperature (via DVFS)Ensure heating needs are satisfiedNeed to reach 100% of platform usage.⇒ An idle QRad is a lack of heatingDispatch instancesSchedule instancesQNodeQBoxQRadClément Mommessin Scheduling at the Edge June 27th, 2019 7 / 23Multiple ObjectivesDifferent objectives for different users:Cloud users: Minimize waiting/completion time of tasksIoT tasks: “Nullify” responsivenessInhabitants: Minimize distance to target temperatureQarnot: Maximize tasks throughput, minimize lack of heatingClément Mommessin Scheduling at the Edge June 27th, 2019 8 / 23Qarnot Solution: Go On-linePeriodic reports (∼30 s) from QBox to QNode with:Number of resources available for each task priorityFree space on diskQNode-scheduling:Sort QBoxes by least available resources first (no temperatureknowledge)Sort tasks by highest priority firstFor each task, dispatch as much instances as possibleQBox-scheduling:Retrieve data-set dependenciesSchedule high priority instances on coolest QRadsSchedule low priority instances on warmest QRadsClément Mommessin Scheduling at the Edge June 27th, 2019 9 / 23Temperature RegulationFrequency and temperature regulator in each QRad:In general: DVFS on QMobos to adapt power consumptionWhen too hot: instances are killed and re-submitted to the QNodeWhen too cold (lack of heating): “background” compute-intensiveinstances are generated (best-effort blockchain mining /)Clément Mommessin Scheduling at the Edge June 27th, 2019 10 / 23Our objectivesMain goal: design, implement and test different placement and schedulingpolicies at both QNode- and QBox-level.Main problem: testing on a production platform is not conceivable andtakes time.⇒ Simulation is what you need!Clément Mommessin Scheduling at the Edge June 27th, 2019 11 / 23Our objectivesMain goal: design, implement and test different placement and schedulingpolicies at both QNode- and QBox-level.Main problem: testing on a production platform is not conceivable andtakes time.⇒ Simulation is what you need!Clément Mommessin Scheduling at the Edge June 27th, 2019 11 / 23Simulation ToolsSimGrid1: Large-scale distributed system simulator with execution andcommunication models.→ Used to simulate platform and tasks execution.Batsim2: Infrascructure simulator for jobs and I/O scheduling.→ Used to drive the simulation, submit tasks and communicate with thedecision process.Pybatsim3: Batsim’s Python API exposing methods to easilycommunicate with the Batsim process.→ Used to implement the QNode and QBox schedulers.1https://github.com/simgrid/simgrid2https://gitlab.inria.fr/batsim/batsim/tree/temperature3https://gitlab.inria.fr/batsim/pybatsim/tree/temperatureClément Mommessin Scheduling at the Edge June 27th, 2019 12 / 23Developed ExtensionsTemperature model: to compute the temperature of the QRad andambient air, based on thermodynamics formulae.External events injector: to replay a machine failure or a temperaturechange.Storage controller: to manage storage entities and data movements.Clément Mommessin Scheduling at the Edge June 27th, 2019 13 / 23Simulation OverviewSimulation process Decision processSimulated platformSimGridBatsimexecutorInstances executorsQNode SchedulerStorage ControllerQBox SchedulerPyBatsimSimGrid kernelInterprocess communicationClément Mommessin Scheduling at the Edge June 27th, 2019 14 / 23Investigating Tasks and Data PlacementsVariants of the QNode dispatcher:StandardLocalityBasedReplicate3LeastLoadedDiskReplicate10LeastLoadedDiskFullReplicate (instantaneous transfers)Standard Qarnot scheduler at QBox-level.1-week simulation inputs from real logs of Qarnot.Clément Mommessin Scheduling at the Edge June 27th, 2019 15 / 23Mean bounded slowdownMean waiting time (s)week1 week2 week3 week405010015020001234WorkloadsSchedulerFullReplicateLocalityBasedReplicate10Replicate3StandardTotal data transfered (GB)week1 week2 week3 week405001000WorkloadsSchedulerFullReplicateLocalityBasedReplicate10Replicate3StandardQBox-level SchedulingList of n instances from Cloud or IoT, with for each instance j:An estimation of work Work jA priority value w jA release date r j (max arrival time of the dependent data-sets)List of m QRads, with for each:A number of QMobosA list of possible speeds of a QMoboThe corresponding power consumption of each speedA diagram of target temperature over a day/a weekThe corresponding power diagram over a day/a weekClément Mommessin Scheduling at the Edge June 27th, 2019 18 / 23QBox-level Scheduling0 24Temperature requiredTime of the dayΔT=0 ΔT=0ΔT=-4 ΔT=+4 ΔT=020°C (home)16°C (work)Target temperature diagramClément Mommessin Scheduling at the Edge June 27th, 2019 19 / 23QBox-level Scheduling0 24Temperature requiredTime of the dayΔT=0 ΔT=0ΔT=-4 ΔT=+4 ΔT=020°C (home)16°C (work)Power required60 W (stable)30 W (stable)200 W (heating)Power diagramClément Mommessin Scheduling at the Edge June 27th, 2019 19 / 23QBox-level SchedulingTHE questions to answer:Where to execute an instance?When to execute an instance?At which speed?⇒ Similar to power capping problemsObjectives:Minimize ∑ j w jC j (where C j is the completion time)Minimize power consumption distance to the power diagramsClément Mommessin Scheduling at the Edge June 27th, 2019 20 / 23Conclusions and ProspectsPreliminary results:Scheduling at the edge depends highly on heterogeneity anddynamicitySimulating edge platforms is possible thanks to Batsim/SimGridData placement is not trivial⇒ Short paper submitted in IEEE Mascots 2019.Current work:Try other placement policies at QNode-level, communicate more withthe QBoxesStudy the QBox theoretical modelValidate the temperature modelClément Mommessin Scheduling at the Edge June 27th, 2019 21 / 23Many ThanksLooking for temperature and power consumption ofmachines/cores in a clusterThis work is supported by the ANR Greco project.Credits: https://www.la-cnem.org/we-need-you/Clément Mommessin Scheduling at the Edge June 27th, 2019 22 / 23Scheduling at the EdgeClément MommessinGiorgio Lucarelli Denis TrystramUniv. Grenoble Alpes, CNRS, INRIA, LIG, F-38000 Grenoble, FranceJune 27th, 2019Clément Mommessin Scheduling at the Edge June 27th, 2019 23 / 23

Scheduling at the Edge

https://hal.inria.fr/hal-02459646/file/slides.pdf

Scheduling at the Edge

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server