The End of Slow Networks: It's Time for a Redesign

Next generation high-performance RDMA-capable networks will require a fundamental rethinking of the design and architecture of modern distributed DBMSs. These systems are commonly designed and optimized under the assumption that the network is the bottleneck: the network is slow and "thin", and thus needs to be avoided as much as possible. Yet this assumption no longer holds true. With InfiniBand FDR 4x, the bandwidth available to transfer data across network is in the same ballpark as the bandwidth of one memory channel, and it increases even further with the most recent EDR standard. Moreover, with the increasing advances of RDMA, the latency improves similarly fast. In this paper, we first argue that the "old" distributed database design is not capable of taking full advantage of the network. Second, we propose architectural redesigns for OLTP, OLAP and advanced analytical frameworks to take better advantage of the improved bandwidth, latency and RDMA capabilities. Finally, for each of the workload categories, we show that remarkable performance improvements can be achieved

Cross-middleware Interoperability in Distributed Concurrent Engineering

Secure, distributed collaboration between different organizations is a key challenge in Grid computing today. The GDCD project has produced a Grid-based demonstrator Virtual Collaborative Facility (VCF) for the European Space Agency. The purpose of this work is to show the potential of Grid technology to support fully distributed concurrent design, while addressing practical considerations including network security, interoperability, and integration of legacy applications. The VCF allows domain engineers to use the concurrent design methodology in a distributed fashion to perform studies for future space missions. To demonstrate the interoperability and integration capabilities of Grid computing in concurrent design, we developed prototype VCF components based on ESA’s current Excel-based Concurrent Design Facility (a non-distributed environment), using a STEP-compliant database that stores design parameters. The database was exposed as a secure GRIA 5.1 Grid service, whilst a .NET/WSE3.0-based library was developed to enable secure communication between the Excel client and STEP database

Research into alternative network approaches for space operations

The main goal is to resolve the interoperability problem of applications employing DOD TCP/IP (Department of Defence Transmission Control Protocol/Internet Protocol) family of protocols on a CCITT/ISO based network. The objective is to allow them to communicate over the CCITT/ISO protocol GPLAN (General Purpose Local Area Network) network without modification to the user's application programs. There were two primary assumptions associated with the solution that was actually realized. The first is that the solution had to allow for future movement to the exclusive use of the CCITT/ISO standards. The second is that the solution had to be software transparent to the currently installed TCP/IP and CCITT/ISO user application programs

Extended Resource Specification Language Reference Manual for ARC versions 0.8 and above

Extended Resource Specification Language Reference Manual for ARC versions 0.8 and abov

A TCP/IP Network Emulator

In this paper a Linux based framework of a TCP/IP emulator is introduced. Several advantages can be noted. Firstly, the maintenance of large numbers of processors is unnecessary. Secondly, compared with simulators constructed with conceptual codes, our emulator framework makes it possible to test the interaction and behaviour of TCP/IP in real Linux network environments. Thirdly, the wired network is fully controlled by a single processor enabling us to separate TCP/IP behaviour over the wireless network, which helps distinguish performance functions that occur due to noisy wireless links. The framework was tested on two Linux processors over an IEEE802.11b wireless link. The simulations show that the complex topology of the heterogeneous network was "realistically" constructed

The End of a Myth: Distributed Transactions Can Scale

The common wisdom is that distributed transactions do not scale. But what if distributed transactions could be made scalable using the next generation of networks and a redesign of distributed databases? There would be no need for developers anymore to worry about co-partitioning schemes to achieve decent performance. Application development would become easier as data placement would no longer determine how scalable an application is. Hardware provisioning would be simplified as the system administrator can expect a linear scale-out when adding more machines rather than some complex sub-linear function, which is highly application specific. In this paper, we present the design of our novel scalable database system NAM-DB and show that distributed transactions with the very common Snapshot Isolation guarantee can indeed scale using the next generation of RDMA-enabled network technology without any inherent bottlenecks. Our experiments with the TPC-C benchmark show that our system scales linearly to over 6.5 million new-order (14.5 million total) distributed transactions per second on 56 machines.Comment: 12 page