Scalability of Distributed Version Control Systems

Distributed version control systems are popular for storing source code, but they are notoriously ill suited for storing large binary files. We report on the results from a set of experiments designed to characterize the behavior of some widely used distributed version control systems with respect to scaling. The experiments measured commit times and repository sizes when storing single files of increasing size, and when storing increasing numbers of single-kilobyte files. The goal is to build a distributed storage system with characteristics similar to version control but for much larger data sets. An early prototype of such a system, Distributed Media Versioning (DMV), is briefly described and compared with Git, Mercurial, and the Git-based backup tool Bup. We find that processing large files without splitting them into smaller parts will limit maximum file size to what can fit in RAM. Storing millions of small files will result in inefficient use of disk space. And storing files with hash-based file and directory names will result in high-latency write operations, due to having to switch between directories rather than performing a sequential write. The next-phase strategy for DMV will be to break files into chunks by content for de-duplication, then re-aggregating the chunks into append-only log files for low-latency write operations and efficient use of disk space

Going Large-scale in P2P Experiments Using the JXTA Distributed Framework

The interesting properties of P2P systems (high availability despite node volatility, support for heterogeneous architectures, high scalability, etc.) make them attractive for distributed computing. However, conducting large-scale experiments with these systems arise as a major challenge. Simulation allows to model only partially the behavior of P2P prototypes. Experiments on real testbeds encounter serious difficulty with large-scale deployment and control of peers. This paper shows that using an optimized version of the JXTA Distributed Framework (JDF) allows to easily deploy, configure and control P2P experiments. We illustrate these features with sample tests performed with our JXTA-based grid data sharing service, for various large-scale configurations

Going Large-scale in P2P Experiments Using the JXTA Distributed Framework

The interesting properties of P2P systems (high availability despite node volatility, support for heterogeneous architectures, high scalability, etc.) make them attractive for distributed computing. However, conducting large-scale experiments with these systems arise as a major challenge. Simulation allows to model only partially the behavior of P2P prototypes. Experiments on real testbeds encounter serious difficulty with large-scale deployment and control of peers. This paper shows that using an optimized version of the JXTA Distributed Framework (JDF) allows to easily deploy, configure and control P2P experiments. We illustrate these features with sample tests performed with our JXTA-based grid data sharing service, for various large-scale configurations

ICONA: Inter Cluster ONOS Network Application

Several Network Operating Systems (NOS) have been proposed in the last few years for Software Defined Networks; however, a few of them are currently offering the resiliency, scalability and high availability required for production environments. Open Networking Operating System (ONOS) is an open source NOS, designed to be reliable and to scale up to thousands of managed devices. It supports multiple concurrent instances (a cluster of controllers) with distributed data stores. A tight requirement of ONOS is that all instances must be close enough to have negligible communication delays, which means they are typically installed within a single datacenter or a LAN network. However in certain wide area network scenarios, this constraint may limit the speed of responsiveness of the controller toward network events like failures or congested links, an important requirement from the point of view of a Service Provider. This paper presents ICONA, a tool developed on top of ONOS and designed in order to extend ONOS capability in network scenarios where there are stringent requirements in term of control plane responsiveness. In particular the paper describes the architecture behind ICONA and provides some initial evaluation obtained on a preliminary version of the tool.Comment: Paper submitted to a conferenc

Improving the scalability of parallel N-body applications with an event driven constraint based execution model

The scalability and efficiency of graph applications are significantly constrained by conventional systems and their supporting programming models. Technology trends like multicore, manycore, and heterogeneous system architectures are introducing further challenges and possibilities for emerging application domains such as graph applications. This paper explores the space of effective parallel execution of ephemeral graphs that are dynamically generated using the Barnes-Hut algorithm to exemplify dynamic workloads. The workloads are expressed using the semantics of an Exascale computing execution model called ParalleX. For comparison, results using conventional execution model semantics are also presented. We find improved load balancing during runtime and automatic parallelism discovery improving efficiency using the advanced semantics for Exascale computing.Comment: 11 figure

Filtering and scalability in the ECO distributed event model

Event-based communication is useful in many application domains, ranging from small, centralised applications to large, distributed systems. Many different event models have been developed to address the requirements of different application domains. One such model is the ECO model which was designed to support distributed virtual world applications. Like many other event models, ECO has event filtering capabilities meant to improve scalability by decreasing network traffic in a distributed implementation. Our recent work in event-based systems has included building a fully distributed version of the ECO model, including event filtering capabilities. This paper describes the results of our evaluation of filters as a means of achieving increased scalability in the ECO model. The evaluation is empirical and real data gathered from an actual event-based system is used