140 research outputs found
How many planet-wide leaders should there be?
Geo-replication becomes increasingly important for modern planetary scale distributed systems, yet it comes with a specific challenge: latency, bounded by the speed of light. In particular, clients of a geo-replicated system must communicate with a leader which must in turn communicate with other replicas: wrong selection of a leader may result in unnecessary round-trips across the globe. Classical protocols such as celebrated Paxos, have a single leader making them unsuitable for serving widely dispersed clients. To address this issue, several all-leader geo-replication protocols have been proposed recently, in which every replica acts as a leader. However, because these protocols require coordination among all replicas, commiting a client's request at some replica may incure the so-called "delayed commit" problem, which can introduce even a higher latency than a classical single-leader majority-based protocol such as Paxos.
In this paper, we argue that the "right" choice of the number of leaders in a geo-replication protocol depends on a given replica configuration and propose Droopy, an optimization for state machine replication protocols that explores the space between single-leader and all-leader by dynamically reconfiguring the leader set. We implement Droopy on top of Clock-RSM, a state-of-the-art all-leader protocol. Our evaluation on Amazon EC2 shows that, under typical imbalanced workloads, Droopy-enabled Clock-RSM efficiently reduces latency compared to native Clock-RSM, whereas in other cases the latency is the same as that of the native Clock-RSM
DRAG: Divergence-based Adaptive Aggregation in Federated learning on Non-IID Data
Local stochastic gradient descent (SGD) is a fundamental approach in
achieving communication efficiency in Federated Learning (FL) by allowing
individual workers to perform local updates. However, the presence of
heterogeneous data distributions across working nodes causes each worker to
update its local model towards a local optimum, leading to the phenomenon known
as ``client-drift" and resulting in slowed convergence. To address this issue,
previous works have explored methods that either introduce communication
overhead or suffer from unsteady performance. In this work, we introduce a
novel metric called ``degree of divergence," quantifying the angle between the
local gradient and the global reference direction. Leveraging this metric, we
propose the divergence-based adaptive aggregation (DRAG) algorithm, which
dynamically ``drags" the received local updates toward the reference direction
in each round without requiring extra communication overhead. Furthermore, we
establish a rigorous convergence analysis for DRAG, proving its ability to
achieve a sublinear convergence rate. Compelling experimental results are
presented to illustrate DRAG's superior performance compared to
state-of-the-art algorithms in effectively managing the client-drift
phenomenon. Additionally, DRAG exhibits remarkable resilience against certain
Byzantine attacks. By securely sharing a small sample of the client's data with
the FL server, DRAG effectively counters these attacks, as demonstrated through
comprehensive experiments
Towards Scaling Blockchain Systems via Sharding
Existing blockchain systems scale poorly because of their distributed
consensus protocols. Current attempts at improving blockchain scalability are
limited to cryptocurrency. Scaling blockchain systems under general workloads
(i.e., non-cryptocurrency applications) remains an open question. In this work,
we take a principled approach to apply sharding, which is a well-studied and
proven technique to scale out databases, to blockchain systems in order to
improve their transaction throughput at scale. This is challenging, however,
due to the fundamental difference in failure models between databases and
blockchain. To achieve our goal, we first enhance the performance of Byzantine
consensus protocols, by doing so we improve individual shards' throughput.
Next, we design an efficient shard formation protocol that leverages a trusted
random beacon to securely assign nodes into shards. We rely on trusted
hardware, namely Intel SGX, to achieve high performance for both consensus and
shard formation protocol. Third, we design a general distributed transaction
protocol that ensures safety and liveness even when transaction coordinators
are malicious. Finally, we conduct an extensive evaluation of our design both
on a local cluster and on Google Cloud Platform. The results show that our
consensus and shard formation protocols outperform state-of-the-art solutions
at scale. More importantly, our sharded blockchain reaches a high throughput
that can handle Visa-level workloads, and is the largest ever reported in a
realistic environment.Comment: This is an updated version of the Chain of Trust: Can Trusted
Hardware Help Scaling Blockchains? paper. This version is to be appeared in
SIGMOD 201
The paleoclimatic footprint in the soil carbon stock of the Tibetan permafrost region
Data and code availability The authors declare that the majority of the data supporting the findings of this study are available through the links given in the paper. The unpublished data are available from the corresponding author upon request. The new estimate of Tibetan soil carbon stock and R code are available in a persistent repository (https://figshare.com/s/4374f28d880f366eff6d). Acknowledgements This study was supported by the Strategic Priority Research Program (A) of the Chinese Academy of Sciences (XDA20050101), the National Natural Science Foundation of China (41871104), Key Research and Development Programs for Global Change and Adaptation (2017YFA0603604), International Partnership Program of the Chinese Academy of Sciences (131C11KYSB20160061) and the Thousand Youth Talents Plan project in China. Jinzhi Ding acknowledges the General (2017M620922) and the Special Grade (2018T110144) of the Financial Grant from the China Postdoctoral Science Foundation.Peer reviewedPublisher PD
State-machine replication for planet-scale systems
Online applications now routinely replicate their data at multiple sites around the world. In this paper we present Atlas, the first state-machine replication protocol tailored for such planet-scale systems. Atlas does not rely on a distinguished leader, so clients enjoy the same quality of service independently of their geographical locations. Furthermore, client-perceived latency improves as we add sites closer to clients. To achieve this, Atlas minimizes the size of its quorums using an observation that concurrent data center failures are rare. It also processes a high percentage of accesses in a single round trip, even when these conflict. We experimentally demonstrate that Atlas consistently outperforms state-of-The-Art protocols in planet-scale scenarios. In particular, Atlas is up to two times faster than Flexible Paxos with identical failure assumptions, and more than doubles the performance of Egalitarian Paxos in the YCSB benchmark.H2020 - Horizon 2020 Framework Programme(825184
SVIP Induces Localization of p97/VCP to the Plasma and Lysosomal Membranes and Regulates Autophagy
The small p97/VCP-interacting protein (SVIP) functions as an inhibitor of the endoplasmic reticulum (ER)-associated degradation (ERAD) pathway. Here we show that overexpression of SVIP in HeLa cells leads to localization of p97/VCP at the plasma membrane, intracellular foci and juxtanuclear vacuoles. The p97/VCP-positive vacuolar structures colocalized or associated with LC3 and lamp1, suggesting that SVIP may regulate autophagy. In support of this possibility, knockdown of SVIP diminished, whereas overexpression of SVIP enhanced LC3 lipidation. Surprisingly, knockdown of SVIP reduced the levels of p62 protein at least partially through downregulation of its mRNA, which was accompanied by a decrease in starvation-induced formation of p62 bodies. Overexpression of SVIP, on the other hand, increased the levels of p62 protein and enhanced starvation-activated autophagy as well as promoted sequestration of polyubiquitinated proteins and p62 in autophagosomes. These results suggest that SVIP plays a regulatory role in p97 subcellular localization and is a novel regulator of autophagy
- …