dReDBox: Materializing a full-stack rack-scale system prototype of a next-generation disaggregated datacenter

Current datacenters are based on server machines, whose mainboard and hardware components form the baseline, monolithic building block that the rest of the system software, middleware and application stack are built upon. This leads to the following limitations: (a) resource proportionality of a multi-tray system is bounded by the basic building block (mainboard), (b) resource allocation to processes or virtual machines (VMs) is bounded by the available resources within the boundary of the mainboard, leading to spare resource fragmentation and inefficiencies, and (c) upgrades must be applied to each and every server even when only a specific component needs to be upgraded. The dRedBox project (Disaggregated Recursive Datacentre-in-a-Box) addresses the above limitations, and proposes the next generation, low-power, across form-factor datacenters, departing from the paradigm of the mainboard-as-a-unit and enabling the creation of function-block-as-a-unit. Hardware-level disaggregation and software-defined wiring of resources is supported by a full-fledged Type-1 hypervisor that can execute commodity virtual machines, which communicate over a low-latency and high-throughput software-defined optical network. To evaluate its novel approach, dRedBox will demonstrate application execution in the domains of network functions virtualization, infrastructure analytics, and real-time video surveillance.This work has been supported in part by EU H2020 ICTproject dRedBox, contract #687632.Peer ReviewedPostprint (author's final draft

On interconnecting and orchestrating components in disaggregated data centers:The dReDBox project vision

Computing systems servers-low-or high-end ones have been traditionally designed and built using a main-board and its hardware components as a 'hard' monolithic building block; this formed the base unit on which the system hardware and software stack design build upon. This hard deployment and management border on compute, memory, network and storage resources is either fixed or quite limited in expandability during design time and in practice remains so throughout machine lifetime as subsystem upgrades are seldomely employed. The impact of this rigidity has well known ramifications in terms of lower system resource utilization, costly upgrade cycles and degraded energy proportionality. In the dReDBox project we take on the challenge of breaking the server boundaries through materialization of the concept of disaggregation. The basic idea of the dReDBox architecture is to use a core of high-speed, low-latency opto-electronic fabric that will bring physically distant components more closely in terms of latency and bandwidth. We envision a powerful software-defined control plane that will match the flexibility of the system to the resource needs of the applications (or VMs) running in the system. Together the hardware, interconnect, and software architectures will enable the creation of a modular, vertically-integrated system that will form a datacenter-in-a-box

A software-defined architecture and prototype for disaggregated memory rack scale systems

Disaggregation and rack-scale systems have the potential of drastically increasing TCO and utilization of cloud datacenters, while maintaining performance. In this paper, we present a novel rack-scale system architecture featuring software-defined remote memory disaggregation. Our hardware design and operating system extensions enable unmodified applications to dynamically attach to memory segments residing on physically remote memory pools and use such remote segments in a byte-addressable manner, as if they were local to the application. Our system features also a control plane that automates software-defined dynamic matching of compute to memory resources, as driven by datacenter workload needs. We prototyped our system on the commercially available Zynq Ultrascale+ MPSoC platform. To our knowledge, this is the first time a software-defined disaggregated system has been prototyped on commercial hardware and evaluated through industry standard software benchmarks. Our initial results - using benchmarks that are artificially highly adversarial in terms of memory bandwidth - show that disaggregated memory access exhibits a round-trip latency of only 134 clock cycles; and a throughput penalty of as low as 55%, relative to locally-attached memory. We also discuss estimations as to how our findings may translate to applications with pragmatically milder memory aggressiveness levels, as well as innovation avenues across the stack opened up by our work

Disaggregated Compute, Memory and Network Systems: A New Era for Optical Data Centre Architectures

The disaggregated dRedBox Data Centre architecture is proposed that enables dynamic allocation of pooled compute and memory resources. An orchestration platform is described and algorithms are simulated that demonstrate the efficient utilization of IT infrastructure

dRedDbox: Demonstrating disaggregated memory in an optical Data Centre

This paper showcases the first experimental demonstration of disaggregated memory using the dRedDbox optical Data Centre architecture. Experimental results demonstrate the 4-tier network scalability and performance of the system at the physical and application layer

MCF-SMF Hybrid Low-Latency Circuit-Switched Optical Network for Disaggregated Data Centers

This paper proposes and experimentally evaluates a fully developed novel architecture with purpose built low latency communication protocols for next generation disaggregated data centers (DDCs). In order to accommodate for capacity and latency needs of disaggregated IT elements (i.e. CPU, memory), this architecture makes use of a low latency and high capacity circuit switched optical network for interconnecting various endpoints, that are equipped with multi-channel Silicon photonic based integrated transceivers. In a move to further decrease the perceived latency between various disaggregated IT elements, this paper proposes a) a novel network topology, which cuts down the latency over the optical network by 34% while enhancing system scalability and b) channel bonding over multicore fiber (MCF) switched links to reduce head to tail latency and in turn increase sustained memory bandwidth for disaggregated remote memory. Furthermore, to reduce power consumption and enhance space efficiency, the integration of novel multi core fiber (MCF) based transceivers, fibers and optical switches are proposed and experimentally validated at the physical layer for this topology. It is shown that the integration of MCF based subsystems in this topology can bring about an improvement in energy efficiency of the optical switching layer which is above 60%. Finally, the performance of this proposed architecture and topology is evaluated experimentally at the application layer where the perceived memory throughput for accessing remote and local resources is measured and compared using electrical circuit and packet switching. The results also highlight a multi fold increase in application perceived memory throughput over the proposed DDC topology by utilization and bonding of multiple optical channels to interconnect disaggregated IT elements that can be carried over MCF links

Demonstration of NFV for mobile edge computing on an optically disaggregated datacentre in a box

This demonstrator showcases the hardware and software integration achieved by the dReDBox project [1] towards realization of a novel architecture using dynamically-reconfigurable optical interconnects to create a flexible, scalable and efficient disaggregated datacentre infrastructure

Supporting multitasking of pipelined computations on embedded parallel processor arrays

This paper presents software support that enables seamless task restructuring and load balancing of pipelined applications at runtime, making it possible to dynamically pick the stages that will be executed as separate tasks on distinct CPUs, depending on the currently available resources and the execution context. This functionality is integrated in a development and execution framework for pipelined applications targeted at reconfigurable (in terms of interconnections), heterogeneous (in terms of architecture and/or clock speed), distributed memory, embedded Parallel Processor Arrays (PPAs). The primary motivation for this work is to support the use of PPA on-chip architectures, which are currently considered as dedicated accelerators, in a multitasking execution context where the available processor cores are distributed among concurrently executing applications. As a proof-of-concept, we discuss the execution of two pipelined applications on an FPGA-based prototype platform that features Xilinx Microblaze soft processor arrays. © 2009 IEEE

On the Implementation of a Software-Defined Memory Control Plane for Disaggregated Datacenters

By adopting a disaggregated hardware architecture, datacenters can achieve considerable efficiency gains and transition to a more sustainable and green future. By decoupling resources from a single monolithic server and connecting them through a high-speed optical network, it is possible to significantly increase resource utilization and reduce power consumption by consolidating workloads into fewer resource units. In this paper, we design and develop a software-defined control plane for disaggregated memory datacenters. Its core component, the Software-Defined Memory Controller, is the orchestrating software, which efficiently materializes the disaggregation concept. It accomplishes this by managing and monitoring remote resource pools, allocating resources to workloads, instrumenting the dynamic configuration of the underlying optical network for interconnecting remote compute and memory resources and interacting with software agents residing in host and guest Operating Systems for coordinating the attachment of remote memory. A major contribution of our design is the minimization of the delay for scheduling workloads and Virtual Machines in a disaggregated datacenter, which is accomplished with the efficient modelling of the disaggregated resources and networking elements into a graph for retrieving configuration data, as well as the optimization of the graph implementation. The architecture is based on the Everything-as-a-Service paradigm and is tightly coupled with OpenStack, the leading cloud infrastructure management software. Evaluation experiments validated the employment of a graph database system by the Software-Defined Memory Controller by demonstrating 58 percent faster query times than relational databases for a small-to-medium-sized datacenter, with the percentage increasing as the size of the datacenter grows. © 2022 IEEE

TLQAP: A topology and link quality assessment protocol for efficient node allocation on wireless testbeds

In this paper we present Topology and Link Quality Assessment Protocol (TLQAP), which we have implemented as a wireless testbed management framework component, that is used to inspect link quality between wireless testbed nodes and appropriately map them to user experiment requirements. TLQAP is mainly an OSI layer 2 design for fixed location, non RF-isolated wireless testbed deployments, which assesses interconnection topology and link quality by estimating packet delivery ratio (PDR) and transmission delay at each node for all requested channel, rate and transmission power combinations. Moreover, TLQAP builds a measurement history log and creates a channel utilization profile, in the context of each testbed node, for all the nearby testbed-external devices that operate independently in the region and are not under the management framework control. The analysis of this information enables TLQAP to choose the channels that have the highest probability of being free during an experiment. TLQAP OSI layer 2 component has been implemented in the click modular router framework and the controller component has been integrated with OMF management framework for wireless testbeds. To outline TLQAP benefits, we have performed experiments on our ORBIT node testbed and we compare it to an existing application level measuring tool. Copyright 2009 ACM