Article thumbnail

Ohio State University (OSU-CISRC-8/07-TR53) OptimizedDistributed DataSharing Substrate in Multi-CoreCommodity Clusters: AComprehensive Study with Applications

By K. Vaidyanathan, P. Lai, S. Narravula and D. K. PandaK. Vaidyanathan, P. Lai, S. Narravula and D. K. Panda

Abstract

Distributed applications tend to have a complex design due to issues such as concurrency, synchronization and communication. Researchers in the past have proposed simpler abstractions to hide these complexities. However, many of the proposed techniques use messaging protocols which incur high overhead and are not very scalable. To address these limitations, in our previous work [20], we proposed an efficient Distributed Data Sharing Substrate (DDSS) using the features of high-speed networks. In this paper, we propose several design optimizations for DDSS in multi-core systems such as the combination of shared memory and message queues for inter-process communication, dedicated thread for communication progress and for onloading DDSS operations such as get and put. Our micro-benchmark results not only show a very low latency in DDSS operations but also demonstrate the scalability of DDSS with increasing number of processes. Application evaluations with R-Tree and B-Tree query processing and distributed STORM shows an improvement of up to 56%, 45 % and 44%, respectively, as compared to the traditional implementations,whileevaluationswithapplicationcheckpointing using DDSS demonstrate the scalability with increasing number of checkpointing applications. Further, in our evaluations, we demonstrate the portability of DDSS across multiple modern interconnects such as InfiniBand and iWARP-capable 10-Gigabit Ethernet networks (applicable for both LAN/WAN environments).

Year: 2012
OAI identifier: oai:CiteSeerX.psu:10.1.1.218.1098
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://citeseerx.ist.psu.edu/v... (external link)
  • http://nowlab.cse.ohio-state.e... (external link)

  • To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.

    Suggested articles