2,322 research outputs found
Exploiting the Synergy Between Gossiping and Structured Overlays
In this position paper we argue for exploiting the synergy between gossip-based algorithms and structured overlay networks (SON). These two strands of research have both aimed at building fault-tolerant, dynamic, self-managing, and large-scale distributed systems. Despite the common goals, the two areas have, however, been relatively isolated. We focus on three problem domains where there is an untapped potential of using gossiping combined with SONs. We argue for applying gossip-based membership for ring-based SONs---such as Chord and Bamboo---to make them handle partition mergers and loopy networks. We argue that small world SONs---such as Accordion and Mercury---are specifically well-suited for gossip-based membership management. The benefits would be better graph-theoretic properties. Finally, we argue that gossip-based algorithms could use the overlay constructed by SONs. For example, many unreliable broadcast algorithms for SONs could be augmented with anti-entropy protocols. Similarly, gossip-based aggregation could be used in SONs for network size estimation and load-balancing purposes
Broadcasting in Prefix Space: P2P Data Dissemination with Predictable Performance
A broadcast mode may augment peer-to-peer overlay networks with an efficient,
scalable data replication function, but may also give rise to a virtual link
layer in VPN-type solutions. We introduce a simple broadcasting mechanism that
operates in the prefix space of distributed hash tables without signaling. This
paper concentrates on the performance analysis of the prefix flooding scheme.
Starting from simple models of recursive -ary trees, we analytically derive
distributions of hop counts and the replication load. Extensive simulation
results are presented further on, based on an implementation within the OverSim
framework. Comparisons are drawn to Scribe, taken as a general reference model
for group communication according to the shared, rendezvous-point-centered
distribution paradigm. The prefix flooding scheme thereby confirmed its widely
predictable performance and consistently outperformed Scribe in all metrics.
Reverse path selection in overlays is identified as a major cause of
performance degradation.Comment: final version for ICIW'0
PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development
This paper describes PlinyCompute, a system for development of
high-performance, data-intensive, distributed computing tools and libraries. In
the large, PlinyCompute presents the programmer with a very high-level,
declarative interface, relying on automatic, relational-database style
optimization to figure out how to stage distributed computations. However, in
the small, PlinyCompute presents the capable systems programmer with a
persistent object data model and API (the "PC object model") and associated
memory management system that has been designed from the ground-up for high
performance, distributed, data-intensive computing. This contrasts with most
other Big Data systems, which are constructed on top of the Java Virtual
Machine (JVM), and hence must at least partially cede performance-critical
concerns such as memory management (including layout and de/allocation) and
virtual method/function dispatch to the JVM. This hybrid approach---declarative
in the large, trusting the programmer's ability to utilize PC object model
efficiently in the small---results in a system that is ideal for the
development of reusable, data-intensive tools and libraries. Through extensive
benchmarking, we show that implementing complex objects manipulation and
non-trivial, library-style computations on top of PlinyCompute can result in a
speedup of 2x to more than 50x or more compared to equivalent implementations
on Spark.Comment: 48 pages, including references and Appendi
A Survey on Wireless Sensor Network Security
Wireless sensor networks (WSNs) have recently attracted a lot of interest in
the research community due their wide range of applications. Due to distributed
nature of these networks and their deployment in remote areas, these networks
are vulnerable to numerous security threats that can adversely affect their
proper functioning. This problem is more critical if the network is deployed
for some mission-critical applications such as in a tactical battlefield.
Random failure of nodes is also very likely in real-life deployment scenarios.
Due to resource constraints in the sensor nodes, traditional security
mechanisms with large overhead of computation and communication are infeasible
in WSNs. Security in sensor networks is, therefore, a particularly challenging
task. This paper discusses the current state of the art in security mechanisms
for WSNs. Various types of attacks are discussed and their countermeasures
presented. A brief discussion on the future direction of research in WSN
security is also included.Comment: 24 pages, 4 figures, 2 table
The Family of MapReduce and Large Scale Data Processing Systems
In the last two decades, the continuous increase of computational power has
produced an overwhelming flow of data which has called for a paradigm shift in
the computing architecture and large scale data processing mechanisms.
MapReduce is a simple and powerful programming model that enables easy
development of scalable parallel applications to process vast amounts of data
on large clusters of commodity machines. It isolates the application from the
details of running a distributed program such as issues on data distribution,
scheduling and fault tolerance. However, the original implementation of the
MapReduce framework had some limitations that have been tackled by many
research efforts in several followup works after its introduction. This article
provides a comprehensive survey for a family of approaches and mechanisms of
large scale data processing mechanisms that have been implemented based on the
original idea of the MapReduce framework and are currently gaining a lot of
momentum in both research and industrial communities. We also cover a set of
introduced systems that have been implemented to provide declarative
programming interfaces on top of the MapReduce framework. In addition, we
review several large scale data processing systems that resemble some of the
ideas of the MapReduce framework for different purposes and application
scenarios. Finally, we discuss some of the future research directions for
implementing the next generation of MapReduce-like solutions.Comment: arXiv admin note: text overlap with arXiv:1105.4252 by other author
- …