Towards Low-Latency Byzantine Agreement Protocols Using RDMA

Byzantine fault tolerance (BFT) protocols can mitigate attacks and errors and are increasingly investigated as consensus protocols in blockchains. However, they are traditionally considered costly in terms of message complexity and latency due to the required multiple rounds of message exchanges. With the availability of Remote Direct Memory Access (RDMA) in data centers, message exchange latency can be reduced compared to TCP, as RDMA enables kernel bypassing and thereby avoids intermediate data copying. Retaining the performance benefits for RDMA during its integration, however, is non-trivial and error-prone. While the use of RDMA has previously been explored for key/value stores, databases and distributed file systems, agreement protocols especially for BFT have so far been neglected. We investigate the usage of RDMA in the Reptor BFT protocol for low-latency agreement and show first steps towards an RDMA-enabled consensus protocol. For this, we present RUBIN, a framework offering similar functionality to the Java NIO selector, which can handle multiple network connections efficiently with a single thread and is employed in several BFT protocol implementations such as BFT-SMART and UpRight

Developing an In-kernel File Sharing Server Solution Based on Server Message Block protocol

Multi-device and multi-service smart environments make heavy use of the Internet and intra-net, thus constantly transferring and saving large amounts of digital data leading to an exponential data growth. This has led to the development of network storage systems such as Storage Area Networks and Network Attached Storage. Network Attached Storage provides a file system level access to data from storage elements that are connected to the network. One of the most widely used protocols in network storage systems, is the Server Message Block(SMB) protocol, that interconnects users from various operating systems such as Windows, Linux and Mac OS. Samba is a popular open-source user-space server that implements the SMB protocol. There have been a multitude of discussions about moving traditional user-space applications like web servers to the kernel-space in order to improve various aspects of the server like CPU utilization, memory utilization, memory footprint, context switching, etc. In this thesis, we have designed and implemented a server in the Linux kernel space. We discuss in detail, the features and functionalities of the newly implemented server. We provide an insight into why some of the design considerations were made, in order to improve the efficiency of protocol handling by the in-kernel file sharing server. We compare the performance of the user-space Samba solution with the in-kernel file sharing solution, implemented and discussed in this thesis, against different workloads to identify the competitiveness of the developed solution. We conclude by discussing what we learned, during the implementation process, along with some ideas for further improving the feature set and performance of the in-kernel server solution

Hyperscsi : Design and development of a new protocol for storage networking

Ph.DDOCTOR OF PHILOSOPH

ShieldBox: Secure Middleboxes using Shielded Execution

Middleboxes that process confidential data cannot be securely deployed in untrusted cloud environments. To securely outsource middleboxes to the cloud, state-of-the-art systems advocate network processing over the encrypted traffic. Unfortunately, these systems support only restrictive functionalities, and incur prohibitively high overheads. This motivated the design of ShieldBox—a secure middlebox framework for deploying high-performance network functions (NFs) over untrusted commodity servers. ShieldBox securely processes encrypted traffic inside a secure container by leveraging shielded execution. More specifically, ShieldBox builds on hardware-assisted memory protection based on Intel SGX to provide strong confidentiality and integrity guarantees. For middlebox developers, ShieldBox exposes a generic interface based on Click to design and implement a wide-range of NFs using its out-of-the-box elements and C++ extensions. For network operators, ShieldBox provides configuration and attestation service for seamless and verifiable deployment of middleboxes. We have implemented ShieldBox supporting important end-to-end features required for secure network processing, and performance optimizations. Our extensive evaluation shows that ShieldBox achieves a near-native throughput and latency to securely process confidential data at line rate

High Performance Computing using Infiniband-based clusters

L'abstract è presente nell'allegato / the abstract is in the attachmen

Building Distributed Systems with Non-Volatile Main Memories and RDMA Networks

High-performance, byte-addressable non-volatile main memories (NVMMs) allow application developers to combine storage and memory into a single layer. These high-performance storage systems would be especially useful in large-scale data center environments where data is distributed and replicated across multiple servers.Unfortunately, existing approaches of providing remote storage access rest on the assumption that storage is slow, so the cost of the software and protocols is acceptable. Such assumption no longer holds for the fast NVMM. As a result, taking full advantage of NVMMs’ potential will require changes in system software and networking protocol. This thesis focuses on accessing remote NVMM efficiently using remote direct memory access (RDMA) network. RDMA enables a client to directly access memory on a remote machine without involving its local CPU.This thesis first presents Mojim, a system that provides replicated, reliable, and highly-available NVMM as an operating system service. Applications can access data in Mojim using normal load and store instructions while controlling when and how updates propagate to replicas using system calls. Our evaluation shows Mojim adds little overhead to the un-replicated system and provides 0.4x to 2.7x the throughput of the un-replicated system.This thesis then presents Orion, a distributed file system designed from for NVMM and RDMA networks. Traditional distributed file systems are designed for slower hard drives. These slower media incentivizes complex optimizations (e.g., queuing, striping, and batching) around disk accesses. Orion combines file system functions and network operations into a single layer. It provides low latency metadata accesses and outperforms existing distributed file systems by a large margin.Finally, an NVMM application can map files backed by an NVMM file system into its address space, and accesses them using CPU instructions. In this case, RDMA and NVMM file systems introduce duplication of effort on permissions, naming, and address translation. We introduce two changes to the existing RDMA protocol: the file memory region (FileMR) and range based address translation. By eliminating redundant translations, FileMR minimizes the number of translations done at the NIC, reducing the load on the NIC’s translation cache and resulting in application performance improvement by 1.8x - 2.0x