International audienceTolerating byzantine faults on a large scale is a challenge: in particular, Desktop Grid environments sustain large numbers of faults that range from crashes to byzantine faults. Solutions in the literature that address byzantine failures are costly and none of them scales to really large numbers of nodes. This paper proposes to distribute task scheduling on trusted nodes in a Cloud network and to have these nodes assess the reliability of worker nodes by means of a reputation system. The resulting architecture is built for scalability and adapts costs to the workload associated with client requests

Arantes, Luciana

Bournat, Marjorie

Friedman, Roy

Marin, Olivier

Sens, Pierre

INRIA a CCSD electronic archive server

HAL Id: hal-01226601https://hal.archives-ouvertes.fr/hal-01226601Submitted on 9 Nov 2015HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Elastic Management of Byzantine FaultsLuciana Arantes, Marjorie Bournat, Roy Friedman, Olivier Marin, Pierre SensTo cite this version:Luciana Arantes, Marjorie Bournat, Roy Friedman, Olivier Marin, Pierre Sens. Elastic Managementof Byzantine Faults. 11th European Dependable Computing Conference (EDCC 2015), Sep 2015,Paris, France. ￿hal-01226601￿Elastic Management of Byzantine FaultsLuciana Arantes∗, Marjorie Bournat∗, Roy Friedman †, Olivier Marin ∗, and Pierre Sens ∗∗Sorbonne Universités, UPMC Univ Paris 06,CNRS, Inria, LIP6F-75005, Paris, FranceEmail: firstname.lastname@lip6.fr†Computer Science DepartmentTechnion - Israel Institute of TechnologyHaifa 32000, IsraelEmail: roy@cs.technion.ac.ilAbstract—Tolerating byzantine faults on a large scale is achallenge: in particular, Desktop Grid environments sustain largenumbers of faults that range from crashes to byzantine faults.Solutions in the literature that address byzantine failures arecostly and none of them scales to really large numbers of nodes.This paper proposes to distribute task scheduling on trustednodes in a Cloud network and to have these nodes assess thereliability of worker nodes by means of a reputation system. Theresulting architecture is built for scalability and adapts costs tothe workload associated with client requests.I. INTRODUCTIONDesktop Grids such as BOINC (Berkley Open Infrastruc-ture for Network Computing) [1] follow a master/worker ap-proach to provide access to an important number of commoditynodes, hence they are well suited for very large computationswhich can be divided into small tasks. However, commoditynodes are not reliable: they are very likely to incur faults thatrange from crashes to byzantine faults. Workers that sustainbyzantine faults may return incorrect results and thereforecorrupt the whole computation.Replication provides an efficient workaround to byzantinefaults: several workers compute the same task concurrently,and if enough workers return the same output it is consideredcorrect. The state of the art distinguishes two categories ofsolutions based on replication: deterministic [2], [3], [4], [5]and probabilistic methods [6], [7]. Deterministic solutionsmake the very strong assumption that there are at most fbyzantine faults in the system, and replicate every task on afixed number of workers. But choosing the right value for fis highly complex. If f is much larger than the real number ofbyzantine faults in the system, then the solution is far from costefficient. Conversely, setting f too small increases the likeli-hood of the solution to return incorrect results. Probabilisticsolutions don’t make any such assumption: instead, they adaptthe degree of replication with a heuristic formula that estimatesthe likelihood of a correct result.Regardless of whether they apply deterministic or prob-abilistic replication, byzantine-tolerant solutions generally donot scale because they rely on a unique and trusted scheduler.In the case of BOINC, a static cluster of trusted schedulers witha consensus protocol handles client queries, which improvesscalability. However, its scalability remains limited becauseof the fixed number of schedulers and it uses a deterministicreplication which is not cost efficient.Fig. 1. Architectural design of our elastic Desktop GridThis position paper presents our proposed approach: anelastic extension of the BOINC desktop grid model on top of acloud service where trusted schedulers use a reputation systemto implement a probabilistic replication scheme. The principleof elasticity is to adapt resource provisioning as a meansto optimize the tradeoff between cost and performance. Oursystem scales in and out as it adapts the number of provisionedschedulers to the number of workers. It also improves costefficiency by maintaining worker reputations and enforcingprobabilistic replication in order to minimize the number ofworkers allocated to each task.II. DESIGN FOR AN ELASTIC DESKTOP GRIDFigure 1 presents our design for a distributed architecturewhere the number of trusted schedulers provisioned in thecloud, called primaries, evolves with the number of availablecommodity nodes that act as workers. Each primary handles asubset of workers, and the intersection between subsets han-dled by two different primaries remains empty. We introducea reliable node, called first-primary, which maintains the listof all active primaries. The role of the first-primary is limitedto dispatching client requests to primaries: a centralized entityshould not hinder scalability. If the first-primary becomes abottleneck, the dispatch can be handled equally well by a largernumber of first-primaries.Our elastic architecture uses probabilistic replication: itmay return incorrect results. Nevertheless it is possible toassess the reliability of a result: each primary maintains adistinct reputation for every worker it handles. A reputation isa subjective estimation of the correctness of previous answersreturned by a worker, and provides some insight about thecorrectness of future answers. A scheduler assigns a defaultreputation value to every new worker that joins its handledsubset. This reputation will evolve: if the worker returns thesame answer as a majority of workers that compute the sametask, then its reputation increases. Conversely its reputationdecreases if its result diverges from the majority answer.A heuristic formula allows every scheduler to calculate thereliability of a task result as a function of the number ofworkers that compute the task and of their reputations.This architectural design enables the application of a widerange of strategies. To start with, there are different waysto update a reputation. For example the reputation value ofa worker can equal the rate of its answers deemed correct.Another possibility is to increase and decrease the reputationvalue with fixed reward and punishment ratios. There are alsomany ways to form replication groups assigned to a task, thegoal being to form a group with a collective reputation highenough to deliver a correct answer with a high probability.For instance the scheduler can just add workers randomlyuntil it achieves its target reliability for the group, or it cansort workers according to their reputations and form smallergroups with the most reputable workers. When a group ofworkers fails to achieve a majority answer, the scheduler mustassign additional workers to this task. This can be done byconsidering the reputation of workers or by considering the gapbetween the number of correct and wrong answers. We havecompiled a collection of such strategies from two previousworks: Sonnek et al. [6] and Arantes et al. [7]. We also intendto apply two incremental replication schemes: iterative andprogressive redundancy [8].The distribution of workers into scheduler subsets offers yetanother range of strategies. Subsets can be arranged to holdworkers with specific ranges of reputation values, or workerreputations can be distributed uniformly among the subsets. Inthe former strategy, clients can request a primary (via the first-primary) in a random way or considering a reputation range.We intend to analyze and compare the performance of ourarchitecture when applying these different strategies.III. ASSESSMENT OF OUR ARCHITECTUREWe identify 5 main metrics for assessing our solution:response time, throughput, reliability, scalability, and costefficiency.We aim for a solution that processes as many clientrequests as possible in the shortest possible amount of time.We associate two metrics with this aim: the average responsetime, that is the time elapsed between the submission of a clientrequest and the reception of the output, and the throughput,the number of requests that get processed within a given timeinterval. We also want a solution that tolerates byzantine faults.We will evaluate its reliability by calculating the percentageof correct answers it outputs. To assess the scalability of oursolution, we will gradually increase the number of workers andsee how it impacts response time and throughput. Finally wewill assess cost efficiency with the average degree of replicationfor every task weighted by the reputation of the allocatedworkers.IV. CONCLUSIONThis paper studies the scalable application of replicationtechniques to tolerate byzantine faults on a Desktop Grid.We introduce a new elastic architecture which scales with thenumber of commodity nodes and uses probabilistic replicationto improve cost efficiency. We also define different strategiesthat can be applied to our architecture, along with metrics toassess these strategies.We have recently finalized a running implementation ofour architecture on top of the SimGrid simulator [9]. Ourimplementation integrates several of the proposed strategiesmentioned in this paper. Our next step is to conduct simulationexperiments in order to analyze and compare the performanceof our architecture when applying these different strategies.ACKNOWLEDGMENTSThis work was supported by a grant from CAMPUS Franceand the Israeli Ministry of Science and Technology PHC-Maimonide, grant #31807PC.REFERENCES[1] D. P. Anderson, “Boinc: A system for public-resource computing andstorage,” in Proc. of the 5th IEEE/ACM International Workshop on GridComputing, Washington, DC, USA, 2004, pp. 4–10.[2] M. Castro and B. Liskov, “Practical byzantine fault tolerance,” inProceedings of the Third USENIX Symposium on Operating SystemsDesign and Implementation (OSDI), USA, 1999, 1999, pp. 173–186.[3] R. Guerraoui, N. Knezevic, V. Quéma, and M. Vukolic, “The next 700BFT protocols,” in Proceedings of the 5th European conference onComputer systems, EuroSys 2010, France, 2010, pp. 363–376.[4] R. Kotla, L. Alvisi, M. Dahlin, A. Clement, and E. L. Wong, “Zyzzyva:speculative byzantine fault tolerance,” in Proceedings of the 21st ACMSymposium on Operating Systems Principles 2007, USA, October 14-17,2007, 2007, pp. 45–58.[5] A. Agbaria and R. Friedman, “A replication- and checkpoint-basedapproach for anomaly-based intrusion detection and recovery,” in 25thInternational Conference on Distributed Computing Systems Workshops,USA, 2005, pp. 137–143.[6] J. D. Sonnek, A. Chandra, and J. B. Weissman, “Adaptive reputation-based scheduling on unreliable distributed infrastructures,” IEEE Trans.Parallel Distrib. Syst., vol. 18, no. 11, pp. 1551–1564, 2007.[7] L. Arantes, R. Friedman, O. Marin, and P. Sens, “Probabilistic ByzantineTolerance for Cloud Computing,” in 34th International Symposium onReliable Distributed Systems (SRDS’15), 2015.[8] Y. Brun, G. Edwards, J. Y. Bang, and N. Medvidovic, “Smart redun-dancy for distributed computation,” in 2011 International Conference onDistributed Computing Systems, ICDCS 2011, USA, 2011, pp. 665–676.[9] H. Casanova, A. Giersch, A. Legrand, M. Quinson, and F. Suter,“Versatile, scalable, and accurate simulation of distributed applicationsand platforms,” Journal of Parallel and Distributed Computing, vol. 74,no. 10, pp. 2899–2917, Jun. 2014.

Elastic Management of Byzantine Faults

HAL Descartes

Hal-Diderot

https://hal.archives-ouvertes.fr/hal-01226601

Elastic Management of Byzantine Faults

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server

HAL Descartes

Hal-Diderot