5 research outputs found
Load Balancing and Virtual Machine Allocation in Cloud-based Data Centers
As cloud services see an exponential increase in consumers, the demand for faster processing of data and a reliable delivery of services becomes a pressing concern. This puts a lot of pressure on the cloud-based data centers, where the consumers’ data is stored, processed and serviced. The rising demand for high quality services and the constrained environment, make load balancing within the cloud data centers a vital concern. This project aims to achieve load balancing within the data centers by means of implementing a Virtual Machine allocation policy, based on consensus algorithm technique. The cloud-based data center system, consisting of Virtual Machines has been simulated on CloudSim – a Java based cloud simulator
Partitioned Paxos via the Network Data Plane
Consensus protocols are the foundation for building fault-tolerant,
distributed systems, and services. They are also widely acknowledged as
performance bottlenecks. Several recent systems have proposed accelerating
these protocols using the network data plane. But, while network-accelerated
consensus shows great promise, current systems suffer from an important
limitation: they assume that the network hardware also accelerates the
application itself. Consequently, they provide a specialized replicated
service, rather than providing a general-purpose high-performance consensus
that fits any off-the-shelf application.
To address this problem, this paper proposes Partitioned Paxos, a novel
approach to network-accelerated consensus. The key insight behind Partitioned
Paxos is to separate the two aspects of Paxos, agreement, and execution, and
optimize them separately. First, Partitioned Paxos uses the network forwarding
plane to accelerate agreement. Then, it uses state partitioning and
parallelization to accelerate execution at the replicas. Our experiments show
that using this combination of data plane acceleration and parallelization,
Partitioned Paxos is able to provide at least x3 latency improvement and x11
throughput improvement for a replicated instance of a RocksDB key-value store
Consensus protocols exploiting network programmability
Services rely on replication mechanisms to be available at all time. The service demanding high availability is replicated on a set of machines called replicas. To maintain the consistency of replicas, a consensus protocol such as Paxos or Raft is used to synchronize the replicas' state. As a result, failures of a minority of replicas will not affect the service as other non-faulty replicas continue serving requests. A consensus protocol is a procedure to achieve an agreement among processors in a distributed system involving unreliable processors. Unfortunately, achieving such an agreement involves extra processing on every request, imposing a substantial performance degradation. Consequently, performance has long been a concern for consensus protocols. Although many efforts have been made to improve consensus performance, it continues to be an important problem for researchers. This dissertation presents a novel approach to improving consensus performance. Essentially, it exploits the programmability of a new breed of network devices to accelerate consensus protocols that traditionally run on commodity servers. The benefits of using programmable network devices to run consensus protocols are twofold: The network switches process packets faster than commodity servers and consensus messages travel fewer hops in the network. It means that the system throughput is increased and the latency of requests is reduced. The evaluation of our network-accelerated consensus approach shows promising results. Individual components of our FPGA- based and switch-based consensus implementations can process 10 million and 2.5 billion consensus messages per second, respectively. Our FPGA-based system as a whole delivers 4.3 times performance of a traditional software consensus implementation. The latency is also better for our system and is only one third of the latency of the software consensus implementation when both systems are under half of their maximum throughputs. In order to drive even higher performance, we apply a partition mechanism to our switch-based system, leading to 11 times better throughput and 5 times better latency. By dynamically switching between software-based and network-based implementations, our consensus systems not only improve performance but also use energy more efficiently. Encouraged by those benefits, we developed a fault-tolerant non-volatile memory system. A prototype using software memory controller demonstrated reasonable overhead over local memory access, showing great promise as scalable main memory. Our network-based consensus approach would have a great impact in data centers. It not only improves performance of replication mechanisms which relied on consensus, but also enhances performance of services built on top of those replication mechanisms. Our approach also motivates others to move new functionalities into the network, such as, key-value store and stream processing. We expect that in the near future, applications that typically run on traditional servers will be folded into networks for performance
Scalable coordination of distributed in-memory transactions
Phd ThesisCoordinating transactions involves ensuring serializability in the presence of concurrent data
accesses. Accomplishing it in an scalable manner for distributed in-memory transactions is the
aim of this thesis work. To this end, the work makes three contributions. It first experimentally
demonstrates that transaction latency and throughput scale considerably well when an atomic
multicast service is offered to transaction nodes by a crash-tolerant ensemble of dedicated nodes
and that using such a service is the most scalable approach compared to practices advocated in
the literature. Secondly, we design, implement and evaluate a crash-tolerant and non-blocking
atomic broadcast protocol, called ABcast, which is then used as the foundation for building the
aforementioned multicast service.
ABcast is a hybrid protocol, which consists of a pair of primary and backup protocols executing
in parallel. The primary protocol is a deterministic atomic broadcast protocol that provides
high performance when node crashes are absent, but blocks in their presence until a group
membership service detects such failures. The backup protocol, Aramis, is a probabilistic protocol
that does not block in the event of node crashes and allows message delivery to continue
post-crash until the primary protocol is able to resume. Aramis design avoids blocking by assuming
that message delays remain within a known bound with a high probability that can be
estimated in advance, provided that recent delay estimates are used to (i) continually adjust
that bound and (ii) regulate flow control. Aramis delivery of broadcasts preserve total order
with a probability that can be tuned to be close to 1. Comprehensive evaluations show that this
probability can be 99.99% or more.
Finally, we assess the effect of low-probability order violations on implementing various
isolation levels commonly considered in transaction systems. These three contributions together
advance the state-of-art in two major ways: (i) identifying a service based approach
to transactional scalability and (ii) establishing a practical alternative to the complex PAXOSiii
style approach to building such a service, by using novel but simple protocols and open-source
software frameworks.Red Ha