Evaluation of data centre networks and future directions

Traffic forecasts predict a more than threefold increase in the global datacentre workload in coming years, caused by the increasing adoption of cloud and data-intensive applications. Consequently, there has been an unprecedented need for ultra-high throughput and minimal latency. Currently deployed hierarchical architectures using electronic packet switching technologies are costly and energy-inefficient. Very high capacity switches are required to satisfy the enormous bandwidth requirements of cloud datacentres and this limits the overall network scalability. With the maturity of photonic components, turning to optical switching in data centres is a viable option to accommodate greater bandwidth and network flexibility while potentially minimising the latency, cost and power consumption. Various DCN architectures have been proposed to date and this thesis includes a comparative analysis of such electronic and optical topologies to judge their suitability based on network performance parameters and cost/energy effectiveness, while identifying the challenges faced by recent DCN infrastructures. An analytical Layer 2 switching model is introduced that can alleviate the simulation scalability problem and evaluate the performance of the underlying DCN architecture. This model is also used to judge the variation in traffic arrival/offloading at the intermediate queueing stages and the findings are used to derive closed form expressions for traffic arrival rates and delay. The results from the simulated network demonstrate the impact of buffering and versubscription and reveal the potential bottlenecks and network design tradeoffs. TCP traffic forms the bulk of current DCN workload and so the designed network is further modified to include TCP flows generated from a realistic traffic generator for assessing the impact of Layer 4 congestion control on the DCN performance with standard TCP and datacentre specific TCP protocols (DCTCP). Optical DCN architectures mostly concentrate on core-tier switching. However, substantial energy saving is possible by introducing optics in the edge tiers. Hence, a new approach to optical switching is introduced using Optical ToR switches which can offer better delay performance than commodity switches of similiar size, while having far less power dissipation. An all-optical topology has been further outlined for the efficient implementation of the optical switch meeting the future scalability demands

Designing data center networks for high throughput

Data centers with tens of thousands of servers now support popular Internet services, scientific research, as well as industrial applications. The network is the foundation of such facilities, giving the large server pool the ability to work together on these applications. The network needs to provide high throughput between servers to ensure that computations are not slowed down by network bottlenecks, with servers waiting on data from other servers. This work address two broad, related questions about high-throughput data center network design: (a) how do we measure and benchmark various network designs for throughput? and (b) how do we design such networks for near-optimal throughput? The problem of designing high-throughput networks has received a lot of attention, with multiple interesting architectures being proposed every year. However, there is no clarity on how one should benchmark these networks and how they compare to each other. In fact, this work shows that commonly used measurement approaches, in particular, cut-metrics like bisection bandwidth, do not predict throughput accurately. In contrast, we directly evaluate the throughput of networks on both uniform and (heretofore unknown) nearly-worst-case traffic matrices, and include here a comparison of 10 networks using this approach. Further, prior work has not addressed a fundamental question: how far are we from throughput-optimal design? In this work, we propose the first upper bound on network throughput for any topology with identical switches. Although designing optimal topologies is infeasible, we demonstrate that random graphs achieve throughput surprisingly close to this bound -- within a few percent at the scale of a few thousand servers for uniform traffic. Our approach also addresses important practical concerns in the design of data center networks, such as incremental expansion and heterogeneous design – as more and varied equipment is added to a data center over the years in response to evolving needs, how do we best accommodate such equipment? Our networks can achieve the same incremental growth at 40% of the expense such growth would incur with past techniques for Clos networks. Further, our approach to designing heterogeneous topologies (i.e., where all the network switches are not identical) achieves 43% higher throughput than a comparable VL2 topology, a heterogeneous network already deployed in Microsoft’s data centers. We acknowledge that the use of random graphs also poses challenges, particularly with regards to efficient routing and physical cabling. We thus present here high-efficiency routing and cabling schemes for such networks as well

Optical Technologies and Control Methods for Scalable Data Centre Networks

Attributing to the increasing adoption of cloud services, video services and associated machine learning applications, the traffic demand inside data centers is increasing exponentially, which necessitates an innovated networking infrastructure with high scalability and cost-efficiency. As a promising candidate to provide high capacity, low latency, cost-effective and scalable interconnections, optical technologies have been introduced to data center networks (DCNs) for approximately a decade. To further improve the DCN performance to meet the increasing traffic demand by using photonic technologies, two current trends are a)increasing the bandwidth density of the transmission links and b) maximizing IT and network resources utilization through disaggregated topologies and architectures. Therefore, this PhD thesis focuses on introducing and applying advanced and efficient technologies in these two fields to DCNs to improve their performance. On the one hand, at the link level, since the traditional single-mode fiber (SMF) solutions based on wavelength division multiplexing (WDM) over C+L band may fall short in satisfying the capacity, front panel density, power consumption, and cost requirements of high-performance DCNs, a space division multiplexing (SDM) based DCN using homogeneous multi-core fibers (MCFs) is proposed.With the exploited bi-directional model and proposed spectrum allocation algorithms, the proposed DCN shows great benefits over the SMF solution in terms of network capacity and spatial efficiency. In the meanwhile, it is found that the inter-core crosstalk (IC-XT) between the adjacent cores inside the MCF is dynamic rather than static, therefore, the behaviour of the IC-XT is experimentally investigated under different transmission conditions. On the other hand, an optically disaggregated DCN is developed and to ensure the performance of it, different architectures, topologies, resource routing and allocation algorithms are proposed and compared. Compared to the traditional server-based DCN, the resource utilization, scalability and the cost-efficiency are significantly improved

A high-speed fault-tolerant interconnect fabric for large-scale multiprocessors

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001.Includes bibliographical references (p. 89-91).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.This thesis describes the design and synthesis of an updated routing block for a nextgeneration wave propagation limited fault-tolerant interconnect fabric for a large-scale shared-memory multiprocessor system. The design is based on the metro multistage interconnection network, and is targeted at minimizing message latency. The design incorporates an efficient new tree-based allocation mechanism and an idempotent messaging protocol. A fat tree topology is the basis for the network. A Verilog implementation of the design is simulated and synthesized into physical hardware, running at speeds as high as 90MHz in an FPGA. Techniques are discussed to vastly improve performance in a potential future design using custom hardware. Further, two potential modifications to the network are considered. First, the performance effect of allocating dedicated physical wires to streamline the idempotent messaging protocol is analyzed. The modification increases the success rate of messages significantly, but the increased latency due to the space taken by the wires overwhelms the potential performance advantage. Second, a scheme for prioritizing messages is developed. This scheme improves the message success rates almost as much as the first modification, reducing the latency of idempotent messages by over 10%. However, this scheme does not increase the number of wires, and has a much smaller overhead. In addition to providing a significant performance advantage, prioritizing messages can help avoid deadlock and livelock situations.by Robert Woods-Corwin.M.Eng

Understanding Quantum Technologies 2022

Understanding Quantum Technologies 2022 is a creative-commons ebook that provides a unique 360 degrees overview of quantum technologies from science and technology to geopolitical and societal issues. It covers quantum physics history, quantum physics 101, gate-based quantum computing, quantum computing engineering (including quantum error corrections and quantum computing energetics), quantum computing hardware (all qubit types, including quantum annealing and quantum simulation paradigms, history, science, research, implementation and vendors), quantum enabling technologies (cryogenics, control electronics, photonics, components fabs, raw materials), quantum computing algorithms, software development tools and use cases, unconventional computing (potential alternatives to quantum and classical computing), quantum telecommunications and cryptography, quantum sensing, quantum technologies around the world, quantum technologies societal impact and even quantum fake sciences. The main audience are computer science engineers, developers and IT specialists as well as quantum scientists and students who want to acquire a global view of how quantum technologies work, and particularly quantum computing. This version is an extensive update to the 2021 edition published in October 2021.Comment: 1132 pages, 920 figures, Letter forma

High speed optical communication systems: From modulation formats to radically new fibres

High volumes of data traffic along with bandwidth hungry applications, such as cloud computing and video on demand, is driving the core optical communication links closer and closer to their maximum capacity. The research community has clearly identifying the coming approach of the nonlinear Shannon limit for standard single mode fibre [1,2]. It is in this context that the work on modulation formats, contained in Chapter 3 of this thesis, was undertaken. The work investigates the proposed energy-efficient four-dimensional modulation formats. The work begins by studying a new visualisation technique for four dimensional modulation formats, akin to constellation diagrams. The work then carries out one of the first implementations of one such modulation format, polarisation-switched quadrature phase-shift keying (PS-QPSK). This thesis also studies two potential next-generation fibres, few-mode and hollow-core photonic band-gap fibre. Chapter 4 studies ways to experimentally quantify the nonlinearities in few-mode fibre and assess the potential benefits and limitations of such fibres. It carries out detailed experiments to measure the effects of stimulated Brillouin scattering, self-phase modulation and four-wave mixing and compares the results to numerical models, along with capacity limit calculations. Chapter 5 investigates hollow-core photonic band-gap fibre, where such fibres are predicted to have a low-loss minima at a wavelength of 2μm. To benefit from this potential low loss window requires the development of telecoms grade subsystems and components. The chapter will outline some of the development and characterisation of these components. The world's first wavelength division multiplexed (WDM) subsystem directly implemented at 2μm is presented along with WDM transmission over hollow-core photonic band-gap fibre at 2μm. References: [1]P. P. Mitra, J. B. Stark, Nature, 411, 1027-1030, 2001 [2] A. D. Ellis et al., JLT, 28, 423-433, 2010