Analysis of Large Unreliable Stochastic Networks

In this paper a stochastic model of a large distributed system where users' files are duplicated on unreliable data servers is investigated. Due to a server breakdown, a copy of a file can be lost, it can be retrieved if another copy of the same file is stored on other servers. In the case where no other copy of a given file is present in the network, it is definitively lost. In order to have multiple copies of a given file, it is assumed that each server can devote a fraction of its processing capacity to duplicate files on other servers to enhance the durability of the system. A simplified stochastic model of this network is analyzed. It is assumed that a copy of a given file is lost at some fixed rate and that the initial state is optimal: each file has the maximum number $d$ of copies located on the servers of the network. Due to random losses, the state of the network is transient and all files will be eventually lost. As a consequence, a transient $d$-dimensional Markov process $(X(t))$ with a unique absorbing state describes the evolution this network. By taking a scaling parameter $N$ related to the number of nodes of the network. a scaling analysis of this process is developed. The asymptotic behavior of $(X(t))$ is analyzed on time scales of the type $t\mapsto N^p t$ for $0\leq p\leq d{-}1$. The paper derives asymptotic results on the decay of the network: Under a stability assumption, the main results state that the critical time scale for the decay of the system is given by $t\mapsto N^{d-1}t$. When the stability condition is not satisfied, it is shown that the state of the network converges to an interesting local equilibrium which is investigated. As a consequence it sheds some light on the role of the key parameters $\lambda$, the duplication rate and $d$, the maximal number of copies, in the design of these systems

SPLAD: scattering and placing data replicas to enhance long-term durability

Distributed storage systems have to ensure data availability and durability despite the occurrence of failures. To do so, many of them rely on replication mechanisms: for each piece of data, several replicas are stored. We show that the layout of the data block copies on the nodes, chiefly the way the copies are scattered, has a major impact on the reparation speed and thus on the data loss ratio. In this paper, we propose SLPAD, an approach that provides the ability: (i) to finely tune the proportion of common content stored by the nodes; and (ii) to control the storage load distribution while creating new data block copies. We propose a simulation model that allows us to present a long-term study of the impact of the data block copies layout and the system load on the data loss ratio.Les systèmes de stockage distribués doivent assurer la disponibilité des données et leur durabilité malgré l'occurrence de défaillances. Pour ce faire, beaucoup d'entre eux utilisent des mécanismes de réplication: pour chaque donnée, plusieurs copies sont stockées. Nous mon- trons que la disposition des copies des données sur les nœuds, surtout la façon dont elles sont dispersées, a un impact majeur sur la vitesse de réparation et donc sur le taux de perte. Dans ce papier, nous proposons SLPAD, une approche qui offre la possibilité: (i) de régler finement la proportion de contenu commun stocké par les nœuds; et (ii) de contrôler la répartition de la charge de stockage lors de la création nouvelles copies. Nous proposons un modèle de simulation qui nous permet de présenter une étude à long terme de l'impact de la disposition des copies des données et de la charge du système sur le taux de perte

An analytical Estimation of Durability in DHTs

International audienceRecent work has shown that the durability of large-scale storage systems such as DHTs can be predicted using a Markov chain model. However, accurate predictions are only possible if the model parameters are also estimated accurately. We show that the Markov chain rates proposed by other authors do not consider several aspects of the system’s behavior, and produce unrealistic predictions. We present a new analytical expression for the chain rates that is condiderably more fine-grain that previous estimations. Our experiments show that the loss rate predicted by our model is much more accurate than previous estimations