Building Distributed Systems with Non-Volatile Main Memories and RDMA Networks

High-performance, byte-addressable non-volatile main memories (NVMMs) allow application developers to combine storage and memory into a single layer. These high-performance storage systems would be especially useful in large-scale data center environments where data is distributed and replicated across multiple servers.Unfortunately, existing approaches of providing remote storage access rest on the assumption that storage is slow, so the cost of the software and protocols is acceptable. Such assumption no longer holds for the fast NVMM. As a result, taking full advantage of NVMMs’ potential will require changes in system software and networking protocol. This thesis focuses on accessing remote NVMM efficiently using remote direct memory access (RDMA) network. RDMA enables a client to directly access memory on a remote machine without involving its local CPU.This thesis first presents Mojim, a system that provides replicated, reliable, and highly-available NVMM as an operating system service. Applications can access data in Mojim using normal load and store instructions while controlling when and how updates propagate to replicas using system calls. Our evaluation shows Mojim adds little overhead to the un-replicated system and provides 0.4x to 2.7x the throughput of the un-replicated system.This thesis then presents Orion, a distributed file system designed from for NVMM and RDMA networks. Traditional distributed file systems are designed for slower hard drives. These slower media incentivizes complex optimizations (e.g., queuing, striping, and batching) around disk accesses. Orion combines file system functions and network operations into a single layer. It provides low latency metadata accesses and outperforms existing distributed file systems by a large margin.Finally, an NVMM application can map files backed by an NVMM file system into its address space, and accesses them using CPU instructions. In this case, RDMA and NVMM file systems introduce duplication of effort on permissions, naming, and address translation. We introduce two changes to the existing RDMA protocol: the file memory region (FileMR) and range based address translation. By eliminating redundant translations, FileMR minimizes the number of translations done at the NIC, reducing the load on the NIC’s translation cache and resulting in application performance improvement by 1.8x - 2.0x

A Methodology for Modelling Mobile Agent-Based Systems (Mobile agent Mobility Methodology - MaMM)

Mobile agents are a particular type of agents that have all the characteristics of an agent and also demonstrate the ability to move or migrate from one node to another in a network environment. Mobile agents have received considerable attention from industry and the research community in recent times due to the fact that their special characteristic of migration help address issues such as network overload, network latency and protocol encapsulation. Due to the current focus in exploiting agent technology mainly in a research environment, there has been an influx of software engineering methodologies for developing multi-agent systems. However, little attention has been given to modelling mobile agents. For mobile agent-based systems to become more widely accepted there is a critical need for a methodology to be developed to address various issues related to modelling mobility of agent . This research study provides an overview of the current approaches, methodologies and modelling languages that can be used for developing multi-agent systems. The overview indicates extensive research on methodologies for modelling multi-agent systems and little on mobility in mobile agent-based systems. An original contribution in this research known as Mobile agent-based Mobility Methodology (MaMM) is the methodology for modelling mobility in mobile agent-based systems using underlying principles of Genetic Algorithms (GA) with emphasis on fitness functions and genetic representation. Delphi study and case studies were employed in carrying out this research

Location based placement of whole distributed systems

The high bandwidth and low latency of the modern internet has made possible the deployment of distributed computing platforms. The XenoServer platform provides a distributed computing plat-form open to all and presents three major new challenges for re-source discovery: Firstly, network location is key for effectively provisioning services, to mitigate against high-latency, high-load or component failure. Secondly, many services require a presence on several servers, with inter-related requirements. Finally, as the platform is open with respect to users and servers, large numbers of queries and updates are expected. To address these requirements we introduce and evaluate Xeno-Search, a new distributed service for selecting the machines to host components of multi-node distributed systems and which is uniquely able to express and efficiently answer complex queries with inter-related location constraints. We demonstrate that Xeno-Search represents a trade-off between accuracy and query time which avoids exhaustive search and supports multiple resources. In addition the performance of the algorithm and the quality of its server selections is investigated and the performance of the dis-tributed service shown to be invariant as the number of nodes or items indexed increases