Some emerging classes of distributed computing systems, such peer-to-peer or grid computing computing systems, are composed of heterogeneous computing resources potentially

unreliable. This paper proposes to use erasure codes to improve the fault-tolerance of parallel distributed computing applications in this context. A general method to generate redundant processes from a set of parallel processes is presented. This scheme allows the recovery of the result of the application even if some of the processes crash

Lacan, Jérôme

English

Open Archive Toulouse Archive Ouverte

Fault-tolerant Distributed Computing Schemebased on Erasure CodesJérôme LacanENSICA1, place E. Blouin, 31053, Toulouse cedex Francejerome.lacan@ensica.frRÉSUMÉ. De nouveaux types de systèmes de calculs répartis tels que les réseaux pair-à-pair oules grilles de calculs sont composés de ressources de calculs hétérogènes dont la fiabilité estégalement fortement variable. Le travail présenté dans ce papier propose de transposer descodes correcteurs à effacement dans le domaine des processus pour améliorer la tolérance auxfautes des systèmes de calculs répartis. La méthode présentée consiste à générer des processusredondants à partir d’un ensemble de processus. Ce système permet de récupérer le résultat descalculs réalisés par l’ensemble des processus d’origine et redondants s’exécutant en parallèlemême si certains d’entre eux connaissent des défaillances par arrêt.ABSTRACT. Some emerging classes of distributed computing systems, such peer-to-peer or gridcomputing computing systems, are composed of heterogeneous computing resources potentiallyunreliable. This paper proposes to use erasure codes to improve the fault-tolerance of paralleldistributed computing applications in this context. A general method to generate redundantprocesses from a set of parallel processes is presented. This scheme allows the recovery of theresult of the application even if some of the processes crash.MOTS-CLÉS : tolérance aux fautes, calculs répartis, codes à effacement, processus redondantsKEYWORDS: Fault tolerance, distributed computing, erasure codes, redundant processes2 Notere’06.1. IntroductionAn increasing number of applications are based on distributed algorithms. Theseapplications can use an heterogeneous network of computing resources with highlyvariable topology and characteristics. The improvement of the fault-tolerance of thedistributed applications is a widely studied problem in this context [CAS 02]. Error-masking techniques are one of the classical approaches to enhance the fault-toleranceof distributed applications. In this paper, we introduce an error-masking techniqueconsisting in adapting the IDA system of Rabin [RAB 89], which disseminates redun-dant data to improve the data availability in distributed storage systems, to distributedcomputing applications. With such a solution, the data will be available despite thecrash of some of the processes/hosts.This paper presents a method to use similar error correcting codes techniques toprotect a set of distributed processes. The idea consists in defining a set of redundantprocesses from a set of given processes so that the failure of some of the given pro-cesses can be compensated by the redundant processes. The main problem we addresshere is how to define redundant processes from a set of processes.Several powerful error correcting codes perform encoding and decoding operationswith only simple XOR operations. The first step of our work consists then in defining acondition on an application in order to "mimic" the XOR operation. Intuitively, if a bitb3 is obtained from two bits b1 and b2 with the XOR operation (i.e. b3 = b1 XOR b2), it ispossible to recover the bit b1 (resp. b2) from b2 and b3 (resp. b1 and b3) with the sameoperation. Let us consider now a distributed application partitioned into 2 processesP1 and P2 with inputs i1 and i2 and outputs o1 and o2. Let us assume that P1 and P2are two instances of a same sub-application P. To reproduce the XOR scheme, we mustdefine an operation on the inputs i1 and i2, called Xin(i1, i2), so that the output o1(resp. o2) can be recovered from o2 and P(Xin(i1, i2)) (resp. o1 and P(Xin(i1, i2))).The operation performing this recovery is called Xout. The applications supportingthis scheme are called XOR-able applications.The main idea is presented in the next Section. In Section 3, the general conditionallowing the definition of the concept of XOR-able application is presented and seve-ral examples of such applications are presented. The classes of error correcting codesusable in this context are presented in Section 4. Section 5 concludes.2. Error correcting codes and distributed applications2.1. Related workFew work have made the connection between Error correcting codes and distribu-ted or parallel applications.The Algorithm-Based Fault-Tolerance (ABFT) techniques [HUA 84] was introdu-ced for parallel processors by Huang and Abraham as a means of error protection forDistributed Computing and Erasure Codes 3parallel matrix operations. The idea was to introduce checksums to provide error de-tection and in some cases, error correction. This approach can be applied efficientlyfor matrix-vector multiplication and for decompositions QR and LU .In the domain of the consensus problem, [MOS 01] proved that if there exists a"relation" between the outputs of the processes, then the consensus problem can besolved for these processes despite a crash of some of them. The connection with errorcorrecting codes is established in [MOS 02].2.2. ContextThe principle of some emerging classes of distributed computing systems, suchpeer-to-peer or grid computing systems, is to connect a wide variety of computertypes and computing resources to create vast "virtual" reservoirs of computers ser-ving geographically widely separated users. These computing resources range fromdesktop workstations to massively parallel processors. However, the heterogeneity ofthe resources makes the performance of the global system highly variable. Classically,this fault-tolerance is ensured by replicating the processes. Several general replication-based strategies were classified to be adapted to the characteristics of the applicationsand the distributed systems [POW 94].A distributed computing system has many similarities with distributed storage sys-tems. Indeed, the problem of the availability of the hosts also arises in distributed sto-rage systems. It can be solved by using data dissemination schemes based on errorcorrecting codes. The idea, proposed by Rabin [RAB 89], consists in splitting the datainto k blocks which are encoded with an erasure code (i.e. an error correcting codedesigned for the erasure channel) to produce a set of n blocks disseminated on n hosts.If the erasure code is optimal, the data is available as long as k hosts among the n onesare alive.The question of the applicability of such a scheme in distributed computing sys-tems is then asked. In fact, a distributed computing system can be considered as asystem providing input data to remote hosts and receiving output data from thesehosts. With this definition, a distributed storage system is simply a distributed compu-ting system where, on each host, the input data are equal to the output data. A naturalway to generalize the fault-tolerant scheme is then to define a code such that 1) theencoding operation processes a vector of input data to generate redundant input data2) the distributed processors process input data to produce output data 3)the decodingoperation processes output data to produce the output data corresponding to the initialinput data. Such a scheme is illustrated in Figure 1. However, compared to classicalutilizations of error correcting codes, the problematic differs in several points. Forexample, by definition, the data are modified by the processes (input and output datadiffer). Moreover, the set of distributed applications can not be specified in terms ofinput and output alphabets (which can be different). The set of applications supportingsuch a scheme is formally defined in Section 3.4 Notere’06.Figure 1. Fault-tolerant scheme in distributed computing systems3. XOR-able applicationsTo define a general strategy to apply error correcting codes techniques to a maxi-mum of applications, we propose in this Section to define an XOR-like operation forprocesses. Note that the goal of this definition is to propose a general but simple me-thod to define redundant processes. It is clear that this definition can be generalizedto applications which does not verify all the hypotheses. Let A be an application par-titioned into k independent processes P1, . . . , Pk. We assume that all the processesperform the same operation, denoted by P, from a set of inputs I to a set of outputs O.Therefore, for any 1 ≤ t ≤ k, each process Pt produces the output ot = P(it) fromthe input it). The output of the application A is then the vector (o1, . . . , ok).Definition 1 The application A is XOR-able if there exists a function :Xin : Ik −→ Iand a set of k functions X(1)out, . . . , X(k)out :X(t)out : Ok −→ O such that for any 1 ≤ t ≤ k,X(t)out(o1, . . . , ot−1, P(Xin(i1, . . . , ik)), ot+1, ok) = ot.Xin denotes the function processing input data andXout denotes the function proces-sing output data.A first class of XOR-able applications are the applications such that P is a linearapplication. Indeed, let us consider an application A partitioned into k processes Pt :Pt :{I −→ Oit −→ ot = P(it)such that P(a.i+ b.i′) = a.P(i)+ b.P(i′), where a and b are any element of a givenfield and where i and i′ are any element of I .Distributed Computing and Erasure Codes 5Proposition 1 The linear application A defined above is XOR-able . Xin and X(t)out,for 1 ≤ t ≤ k, can be defined as follows :∣∣∣∣∣∣Xin : (i1, . . . , ik) −→ Xin(i1, . . . , ik) =∑t=1...,k it andX(t)out :(o1, . . . , ot−1, P(Xin(i1, . . . , ik)), ot+1, . . . , ok) −→P(Xin(i1, . . . , ik))−∑u=1,...,k;u 6=t ouThis proposition can be easily proved by considering that P(Xin(i1, . . . , ik)) is equalto∑t=1...,k P (it) since P is a linear application. Moreover, by definition, ot = P (it)for t = 1, . . . , k. It follows that P(Xin(i1, . . . , ik))−∑u=1,...,k;u 6=t ou is equal to ot.A famous example of distributed linear applications is [KOR 01] where the maintask of the distributed processes consists in performing Discrete Fourier Transforms,which are linear applications. Several more complex applications, e.g. related to li-near algebra, can be considered as XOR-able applications (e.g. computing the inverseof k nonsingular square matrices, computing the determinant of several nonsingularmatrices). Another class of XOR-able applications are some classical cryptographicoperations such the operation P (x) = gx%p. Actually, even if the way to express agiven distributed application as an XOR-able application is not always direct, it ap-pears that this technique can be applied to numerous applications (or sometimes onsome parts of these applications).4. Error correcting codes for XOR-able applicationsWe consider here that the distributed processes crash or provide a correct result.The corresponding channel in terms of error correction is then the erasure channel. Byconstruction, the XOR-able applications reproduce the scheme of the Single ParityCheck (SPC) codes which protects k symbols with a additional symbol defined asthe opposite of the sum of the k first symbols. It follows that this codes and all thecodes derived from SPC can be used here. For example, a product code of severalSPC, which reaches a high level of performance in term of correction capability (see[KOU 02]), can be direcly used to protect XOR-able applications. Note that thesecodes are able to correct more than one error per block.More generally, any binary linear code whose the decoding only uses binary sumscan be used for Several classical classes of binary linear error correcting codes can beused in this context. Reed-Muller, LDPC and "rateless" codes [LUB 02] belong to thisclass of codes.5. ConclusionThis paper has presented a fault-tolerant distributed computing scheme based onerror correcting codes. This scheme allows supporting the crash of some nodes. Toapply error correcting codes concepts to processes, the notion of XOR-able applica-6 Notere’06.tion was introduced and several examples of XOR-able applications were presented.Several classical error correcting codes were proposed to be used in this context.Several problems are opened by this work. The first one is to verify whether it ispossible to apply the proposed fault-tolerant scheme to given distributed applicationssuch that, for example, classical matrix operations. Another problem concerns errorcorrecting codes and more precisely the construction of efficient codes for distribu-ted computing. Finally, the implementation of the fault-tolerant scheme in distributedsystems opens several problems related to the consensus, the distributed decodingstrategy or the dynamic management of redundant computing nodes.AcknowledgmentsThe author thanks Tanguy Pérennou, Jérôme Fimes, Michel Salaün, et al. for theircomments and suggestions.6. Bibliographie[CAS 02] CASANOVA H., « Distributed Computing Research Issues in Grid Computing »,Quarterly Newsletter for the ACM Special Interest Group on Algorithms and ComputationTheory (SIGACT News), vol. 33, no 2, 2002.[HUA 84] HUANG K. H., ABRAHAM J. A., « Algorithm-based fault tolerance for matrix ope-rations », IEEE Trans. on Computers, vol. C-33, 1984, p. 518-528.[KOR 01] KORPELA E., WERTHIMER D., ANDERSON D., COBB J., LEBOFSKY M.,« SETI@home :Massively Distributed Computing for SETI », Computing in Science andEngineering, vol. 3, no 1, 2001, p. 78-83.[KOU 02] KOUSA M., « A novel approach for evaluating the performance of SPC productcodes under erasure decoding », IEEE Transactions on Communications, vol. 50, no 1,2002, p. 7 - 11.[LUB 02] LUBY M., « LT Codes », IEEE Symposium on Foundations of Computer Science,2002, p. 271-280.[MOS 01] MOSTEFAOUI A., RAJSBAUM S., RAYNAL M., « Conditions on Input Vectorsfor Consensus Solvability in Asynchronous Distributed Systems », ACM Symposium onTheory of Computing (STOC’01), ACM, jul. 2001, p. 153-162.[MOS 02] MOSTEFAOUI A., RAJSBAUM S., RAYNAL M., « Asynchronous InteractiveConsistency and its Relation with Error-Correcting Codes », ACM Symposium on Prin-ciples of Distributed Computing (PODC-02), jul. 2002, page 253.[POW 94] POWELLS D., « Distributed fault-tolerance - Lessons learnt from Delta 4 », IEEEMicro., vol. 14, no 1, 1994, p. 36-47.[RAB 89] RABIN M. O., « Efficient dispersal of information for security, load balancing, andfault tolerance », J. ACM, vol. 36, no 2, 1989, p. 335–348, ACM Press.

Fault-tolerant distributed computing scheme based on erasure codes

http://oatao.univ-toulouse.fr/3148/1/Lacan_3148.pdf

Fault-tolerant distributed computing scheme based on erasure codes

Abstract

Similar works

Full text

Available Versions

Open Archive Toulouse Archive Ouverte