1,904 research outputs found
Simplified Distributed Programming with Micro Objects
Developing large-scale distributed applications can be a daunting task.
object-based environments have attempted to alleviate problems by providing
distributed objects that look like local objects. We advocate that this
approach has actually only made matters worse, as the developer needs to be
aware of many intricate internal details in order to adequately handle partial
failures. The result is an increase of application complexity. We present an
alternative in which distribution transparency is lessened in favor of clearer
semantics. In particular, we argue that a developer should always be offered
the unambiguous semantics of local objects, and that distribution comes from
copying those objects to where they are needed. We claim that it is often
sufficient to provide only small, immutable objects, along with facilities to
group objects into clusters.Comment: In Proceedings FOCLASA 2010, arXiv:1007.499
P2CP: A New Cloud Storage Model to Enhance Performance of Cloud Services
This paper presents a storage model named Peer to Cloud and Peer (P2CP). Assuming that the P2CP model follows the Poisson process or Little’s law, we prove that the speed and availability of P2CP is generally better than that of the pure Peer to Peer (P2P) model, the Peer to Server, Peer (P2SP) model or the cloud model. A key feature of our P2CP is that it has three data transmission tunnels: the cloud-user data transmission tunnel, the clients’ data transmission tunnel, and the common data transmission tunnel. P2CP uses the cloud storage system as a common storage system. When data transmission occurs, the data nodes, cloud user, and the non-cloud user are all together involved to complete the transaction
A replicated file system for Grid computing
To meet the rigorous demands of large-scale data sharing in global collaborations, we present a replication scheme for NFSv4 that supports mutable replication without sacrificing strong consistency guarantees. Experimental evaluation indicates a substantial performance advantage over a single-server system. With the introduction of a hierarchical replication control protocol, the overhead of replication is negligible even when applications mostly write and replication servers are widely distributed. Evaluation with the NAS Grid Benchmarks demonstrates that our system provides comparable and often better performance than GridFTP, the de facto standard for Grid data sharing. Copyright © 2008 John Wiley & Sons, Ltd.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/60228/1/1286_ftp.pd
Storage Solutions for Big Data Systems: A Qualitative Study and Comparison
Big data systems development is full of challenges in view of the variety of
application areas and domains that this technology promises to serve.
Typically, fundamental design decisions involved in big data systems design
include choosing appropriate storage and computing infrastructures. In this age
of heterogeneous systems that integrate different technologies for optimized
solution to a specific real world problem, big data system are not an exception
to any such rule. As far as the storage aspect of any big data system is
concerned, the primary facet in this regard is a storage infrastructure and
NoSQL seems to be the right technology that fulfills its requirements. However,
every big data application has variable data characteristics and thus, the
corresponding data fits into a different data model. This paper presents
feature and use case analysis and comparison of the four main data models
namely document oriented, key value, graph and wide column. Moreover, a feature
analysis of 80 NoSQL solutions has been provided, elaborating on the criteria
and points that a developer must consider while making a possible choice.
Typically, big data storage needs to communicate with the execution engine and
other processing and visualization technologies to create a comprehensive
solution. This brings forth second facet of big data storage, big data file
formats, into picture. The second half of the research paper compares the
advantages, shortcomings and possible use cases of available big data file
formats for Hadoop, which is the foundation for most big data computing
technologies. Decentralized storage and blockchain are seen as the next
generation of big data storage and its challenges and future prospects have
also been discussed
Currency management system: a distributed banking service for the grid
Market based resource allocation mechanisms require mechanisms to regulate and manage the usage of traded resources. One mechanism to control this is the definition of some kind of currency. Within this context, we have implemented a first prototype of our Currency Management System, which stands for a decentralized and scalable banking service for the Grid. Basically, our system stores user accounts within a DHT and its basic operation is the transferFunds which, as its name suggests, transfers virtual currency from an account to one another
From Events to Reactions: A Progress Report
Syndicate is a new coordinated, concurrent programming language. It occupies
a novel point on the spectrum between the shared-everything paradigm of threads
and the shared-nothing approach of actors. Syndicate actors exchange messages
and share common knowledge via a carefully controlled database that clearly
scopes conversations. This approach clearly simplifies coordination of
concurrent activities. Experience in programming with Syndicate, however,
suggests a need to raise the level of linguistic abstraction. In addition to
writing event handlers and managing event subscriptions directly, the language
will have to support a reactive style of programming. This paper presents
event-oriented Syndicate programming and then describes a preliminary design
for augmenting it with new reactive programming constructs.Comment: In Proceedings PLACES 2016, arXiv:1606.0540
Automatic Reconfiguration for Large-Scale Reliable Storage Systems
Byzantine-fault-tolerant replication enhances the availability and reliability of Internet services that store critical state and preserve it despite attacks or software errors. However, existing Byzantine-fault-tolerant storage systems either assume a static set of replicas, or have limitations in how they handle reconfigurations (e.g., in terms of the scalability of the solutions or the consistency levels they provide). This can be problematic in long-lived, large-scale systems where system membership is likely to change during the system lifetime. In this paper, we present a complete solution for dynamically changing system membership in a large-scale Byzantine-fault-tolerant system. We present a service that tracks system membership and periodically notifies other system nodes of membership changes. The membership service runs mostly automatically, to avoid human configuration errors; is itself Byzantine-fault-tolerant and reconfigurable; and provides applications with a sequence of consistent views of the system membership. We demonstrate the utility of this membership service by using it in a novel distributed hash table called dBQS that provides atomic semantics even across changes in replica sets. dBQS is interesting in its own right because its storage algorithms extend existing Byzantine quorum protocols to handle changes in the replica set, and because it differs from previous DHTs by providing Byzantine fault tolerance and offering strong semantics. We implemented the membership service and dBQS. Our results show that the approach works well, in practice: the membership service is able to manage a large system and the cost to change the system membership is low
- …