Passive optical networks (PON) technology is increasingly becoming an attractive solution in modern data centres as it provides energy efficient, high capacity, low cost, scalable and flexible connectivity. In this paper we report the implementation of a PON based data centre architecture that provides high resilience and high speed interconnections by providing alternative communication routes between servers in different racks. Each rack is divided into several groups of servers and connects to other racks and the Optical Line Terminal (OLT) through a set of server that acts as relay servers. We implement the switching and routing functionalities within servers using 4×10GE Xilinx NetFPGA, and demonstrate end-to-end communication using IP cameras live video streaming over up to 100 km optical connections through WDM nodes and the PON network

Al-Quzweeni, A

Elmirghani, JMH

Eltraify, AEA

Musa, MOI

arXiv

White Rose Research Online

This is a repository copy of Experimental Evaluation of Passive Optical Network Based Data Centre Architecture.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/134752/Version: Accepted VersionProceedings Paper:Eltraify, AEA, Musa, MOI, Al-Quzweeni, A et al. (1 more author) (2018) Experimental Evaluation of Passive Optical Network Based Data Centre Architecture. In: ICTON 2018. ICTON 2018: 20th International Conference on Transparent Optical Networks, 01-05 Jul 2018, Bucharest, Romania. IEEE . ISBN 978-1-5386-6605-0 https://doi.org/10.1109/ICTON.2018.8473646© 2018 IEEE. This is an author produced version of a paper published in ICTON 2018. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material foradvertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Uploaded in accordance with the publisher's self-archiving policy.eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Experimental Evaluation of Passive Optical Network Based Data Centre Architecture Azza E. A. Eltraify, Mohamed O. I. Musa, Ahmed Al-Quzweeni and Jaafar M.H. Elmirghani  School of Electronic and Electrical Engineering, University of Leeds, Leeds, LS2 9JT, UK  E-mail: {scaeae, m.musa, ml13anaq, j.m.h.elmirghani}@leeds.ac.uk  ABSTRACT Passive optical networks (PON) technology is increasingly becoming an attractive solution in modern data centres as it provides energy efficient, high capacity, low cost, scalable and flexible connectivity. In this paper we report the implementation of a PON based data centre architecture that provides high resilience and high speed interconnections by providing alternative communication routes between servers in different racks. Each rack is divided into several groups of servers and connects to other racks and the Optical Line Terminal (OLT) through a set of server that acts as relay servers. We implement the switching and routing functionalities within servers using 4x10GE Xilinx NetFPGA, and demonstrate end-to-end communication using IP cameras live video streaming over up to 100 km optical connections through WDM nodes and the PON network. Keywords: Passive optical Networks, NetFPGA, ONU, PON, NIC, WDM 1. INTRODUCTION The increasing demand for higher data rates and power efficiency in data centres have been the focus of research and development in data centres in recent years. Several studies have been carried out in optimising energy efficiency and architectures in data centres and core networks connecting them [1] - [10]. However, there is a need to transform these theoretical advancements into practical and tangible systems.  Our previously designed PON based data centre network architecture shown in Figure 1 [11] proposed a server centric architecture that facilitates high speed interconnections among servers by dividing the whole data centre into a set of PON cells that contain an aggregation of several servers, ONUs and an OLT. Inside each cell, servers are divided into racks and within a rack, servers are grouped in multiple groups. Servers of different groups in the same rack are connected via an optical backplane [12]. One of the servers¶ groups is connected to an OLT DQG LV UHVSRQVLEOH IRU WKH UDFN¶V FRPPXQLFDWLRQ ZLWK WKH 2LT, while other groups are connected to other racks in the cell. Inter-rack communication is established by using one of the JURXS¶VVHUYHUVvia a direct connection with the rack of the destination server. Connections between the OLT and racks can be provisioned either using a coupler (TDM) or via an AWGR (WDM) to provide more bandwidth [11], [13] - [15]. One of the major problems in data centres nowadays is keeping up with the increasing demand for higher data rates, faster processing and scalability. Several applications have been implemented to mitigate these problems and help fulfil the massive demand in data rates, one of the solutions was the introduction of NetFPGAs, a Field Programmable Gate Arrays (FPGA) based networking solution. FPGAs were introduced to accelerate the switching, routing and processing of network data. The NetFPGA Platform is an open source hardware and software platform which contains a large Xilinx FPGA that is programmed with user-defined logic and implements the PCI interface to a host processor. The platform includes Static RAMs (SRAMs) that operate synchronously with the FPGA as well as Double Data Rate (DDR2) SDRAM devices that operate asynchronously with the FPGA. A quad-port physical-layer transceiver (PHY) is provided enabling the platform to send and receive packets. The NetFPGA platform now has three main operating boards that are supported by the NetFPGA community which are NetFPGA-SUME, NetFPGA-1G-CML and the NetFPGA-10G [16]. In this paper the NetFPGA-10G is used as a Network Interface Card (NIC) for the servers. It is an FPGA-based PCI Express board with four 10-Gigabit SFP+ interfaces, an [ JHQ 3&,H DGDSWHU FDUG LQFRUSRUDWLQJ ;LOLQ[¶VVirtex-5 TX240TFPGA. It is considered to be ideal for high bandwidth applications [16]. This work aims to experimentally demonstrate the implementation of the proposed data centre architecture and furthermore demonstrate end-to-end communication of different data centres over long distances through various platforms. The end-to-end connectivity is shown using IP cameras live video streaming over up to 100 km optical connections through WDM nodes and the PON network. 2. ARCHITECTURE IMPLEMENTATION To implement the architecture shown in Figure 1, three racks were employed. Each set has three servers grouped together representing a different rack/ data centre. The communication between the servers within the same rack uses a 10 Gbps Cisco Switch and the communication between racks is set up using an optical link where media converters are used to convert the electrical signal to optical. The communication between servers and the flow of data between different racks is illustrated in Figure 2. Figure 3 shows the experimental implementation of the proposed architecture. As an initial implementation, we employ an electronic switch instead of the optical polymer backplane and use the operating system of the servers to forward packets.     Figure 1. Server Centric PON based data centre network architecture [17]   Servers were set up using two different operating systems. Each operating system serves a different purpose. The majority of the servers operated under the open source Debian based Ubuntu Operating system commonly used in networking. Servers acting as gateways operated using MikroTik OS; a specialized Linux Kernel operating system that provides routing functionalities in order to oversee packet routing from and to racks. The IP network is organized into three networks based on the number of racks and communication links between these racks. The connections between servers utilise various paths through different racks to ensure reliability. Hence, in order to realise the data connectivity between servers in different racks, the packet routing has been configured in servers by the use of MikroTik. Figure 2 illustrates the networks and IP assignment where each rack has a server that oversees packet routing from and to the rack configured with two IP addresses, one for intra-rack communication and another for inter-rack communication. Other servers in the rack have been assigned a gateway IP address for inter-rack communication.                    To facilitate communication between multiple data centres located in multiple cities, an IP over DWDM core network, spanning 100 km over optical fibre links, was implemented as shown in Figure 4. Each core node is an MRV/ADVA DWDM node with up to 100 Gbps capabilities. Multiplexers/Demultiplexers are used on the C-band supporting 80 wavelengths, and EDFAs and dispersion compensators are used to improve the quality of the optical signal over long distances. This setup allowed testing data transmission and delay over various long distances representing actual distances between cities in a typical core network. [18].  Figure 2. Intra-rack and inter-rack communication in the proposed PON based data centre architecture Figure 3. Server Centric Architecture Experimental Implementation     Figure 4. IP/WDM Core Network Nodes In order to demonstrate end-to-end communication, all previous set ups were connected together optically as shown in Figure 5, where the IPTV Camera is connected to the proposed architecture and streams live video through the Fog data centre architecture, over the IP/WDM Core networks nodes, and finally to the OLT/ONU where a display is connected to show the video stream.  Figure 5. The implemented end-to-end System of the Proposed Server Centric Data Centre Architecture, IP/WDM Core Network Nodes and OLT/ONU 3. RESULTS In order to achieve the full architecture setup, three components were implemented: (a) Servers group setup (b) IP/WDM Core Network Nodes, and (c) OLT/ONU. The end-to end system connectivity was demonstrated using real time video streaming over a distance of over 100 km. The video was streamed using two different platforms, an IoT camera and another IPTV camera where both resulted in real time high quality live video stream.    Figure 6. The Round Trip time of all nodes in the system for 10 iterations. Network connections and packet routing were tested using the ICMP protocol and route tracing. ICMP signals were sent from each server to all servers and were successfully received with 0% packet loss. Traceroute was sent from the server where the camera was connected in the Fog data centre to the end node in the core network, in order to measure the latency in each hop. The test was performed by sending packets and measuring the round trip time (RTT) of each hop. This signal was sent 10 times, each iteration resulting in different results for the RTT of 150 results in total. Figure 6 shows WKHWHVWVLWHUDWLRQVDQGWKHUHVXOWVRIHDFKKRS¶Vaverage RTT.  The system demonstrated low latency, with RTT less than 1 ms (ranging from 0.144 ms to 0.857ms) as shown in Figure 6, which is significantly low considering the long distance and the difference in platforms.  4. CONCLUSIONS This paper has described an experimental evaluation of a proposed Service Centric PON based data centre architecture that is cost and power efficient. The performance was demonstrated through the use of ICMP signals and a video stream over the server centric architecture, IP/WDM Nodes and OLT/ONU communication. The use of NetFPGA boards for NIC communication provided high speed and high data rate optical interconnections between servers. Future implementation includes adding an Arrayed waveguide grating router (AWGR) to provide wavelength routing between servers [14], [15] leading to a further reduction in cost and power consumption. ACKNOWLEDGEMENTS The authors would like to acknowledge funding from the Engineering and Physical Sciences Research Council (EPSRC), INTERNET (EP/H040536/1) and STAR (EP/K016873/1). All data are provided in full in the results section of this paper. REFERENCES [1] X. Dong, T. El-*RUDVKL DQG - 0 (OPLUJKDQL ³,3 RYHU :'0 QHWZRUNV HPSOR\LQJ UHQHZDEOH HQHUJ\VRXUFHV´J. Lightwave Technol., vol. 29, no. 1, pp. 3±14, 2011.  [2] X. Dong, T. El-*RUDVKLDQG-0+(OPLUJKDQL³*UHHQ,3RYHU:'0QHWZRUNVZLWKGDWDFHQWHUV´J. Lightwave Technol., vol. 29, no. 12, pp. 1861±1880, June 2011.  [3] X. Dong, T. E. H. El-*RUDVKL DQG-0+(OPLUJKDQL ³2Q WKHHQHUJ\HIILFLHQF\RISK\VLFDO WRSRORJ\GHVLJQIRU,3RYHU:'0QHWZRUNV´J. Lightwave Technol., vol. 30, no. 12, pp. 1931±1936, 2012.  [4] A. Lawey, T. El-*RUDVKLDQG-(OPLUJKDQL³'LVWULEXWHGHQHUJ\HIILFLHQWFORXGVRYHUFRUHQHWZRUNV´ J. Lightwave Technol., vol. 32, no. 7, pp. 1261±1281, Apr. 2014. [5] N. Osman, T. El-*RUDVKL / .UXJ DQG - (OPLUJKDQL ³(QHUJ\-efficient future high-GHILQLWLRQ 79´ J. Lightwave Technol., vol. 32, no. 13, pp. 2364±2381, July 2014.  [6] A. Lawey, T. El-*RUDVKL DQG - (OPLUJKDQL ³%LW7RUUHQW FRQWHQW GLVWULEXWLRQ LQ RSWLFDO QHWZRUNV´ J. Lightwave Technol., vol. 32, no. 21, pp. 4209±4225, Nov. 2014. [7] L. Nonde, T. El-*RUDVKL DQG - (OPLUJKDQL ³(QHUJ\ HIILFLHQW YLUWXDO QHWZRUN HPEHGGLQJ IRU FORXGQHWZRUNV´J. Lightwave Technol., vol. 33, no. 9, pp. 1828±1849, May 2015.  [8] 0 0XVD 7 (OJRUDVKL DQG - (OPLUJKDQL ³(QHUJ\ HIILFLHQW VXUYLYDEOH ,3-over-WDM networks with QHWZRUNFRGLQJ´J. Opt. Commun. Netw., vol. 9, no. 3, pp. 207±217, 2017. [9] Musa, Mohamed, Taisir ElgRUDVKL DQG -DDIDU (OPLUJKDQL ³Bounds for Energy-Efficient Survivable IP Over WD01HWZRUNV:LWK1HWZRUN&RGLQJ´ Journal of Optical Communications and Networking 10.5 (2018): 471-481, 2018. [10] Elmirghani, J. M. H., T. Klein, K. Hinton, L. Nonde, A. Q. Lawey, T. E. H. El-Gorashi, M. O. I. Musa, and ; 'RQJ ³GreenTouch GreenMeter core network energy-efficiency improvement measures and optimization´ Journal of Optical Communications and Networking 10, no. 2 (2018): A250-A269. [11] Ali Abdullah Hammadi: ³Future PON Data Centre Networks´, University of Leeds, School of Electronic and Electrical Engineering, Aug. 2016. [12] J. Beals IV, N. Bamiedakis, A. Wonfor, R. Penty, I. White, J. DeGroot Jr, et al.: ³A terabit capacity passive polymer optical backplane based on a novel meshed waveguide architecture´, Applied Physics A, vol. 95, pp. 983-988, 2009. [13] Hammadi, Ali, Taisir EH El-Gorashi, Mohamed OI Musa, and JaafaU 0+ (OPLUJKDQL ³Server-centric PON data cHQWHU DUFKLWHFWXUH´ In Transparent Optical Networks (ICTON), 2016 18th International Conference on, pp. 1-4. IEEE, 2016. [14] A. Hammadi, Mohamed O. I. Musa, T. E. H. El-GRUDVKLDQG-0+(OPLUJKDQL³Resource Provisioning for Cloud PON AWGR-Based Data CentHU $UFKLWHFWXUH´, 21st European Conference on Network and Optical Communications (NOC), Portugal, 2016. [15] Hammadi, Ali, Taisir EH El-Gorashi, and Jaafar MH Elmirghani. ³High performance AWGR PONs in data centre networNV´ In Transparent Optical Networks (ICTON), 2015 17th International Conference on, pp. 1-5. IEEE, 2015. [16]  John W. Lockwood, Nick McKeown, Greg Watson, Glen Gibb, Paul Hartke, Jad Naous, Ramanan 5DJKXUDPDQDQG-LDQ\LQJ/XR³1HW)3*$ - An Open Platform for Gigabit-rate Network Switching and 5RXWLQJ´MSE 2007, San Diego, June 2007. [17] A. Hammadi, T. E. El-GoUDVKL DQG - 0 + (OPLUJKDQL ³PONs in Future Cloud Data Centers´ IEEE Communications Magazine. [18] X. Dong, T. El-*RUDVKL-0+(OPLUJKDQL³Green IP Over WDM Networks With Data Centers´, IEEE Journal of Lightwave Technology, vol.29, no.12, pp.1861-1880, June 2011. 

Experimental Evaluation of Passive Optical Network Based Data Centre Architecture

https://eprints.whiterose.ac.uk/134752/1/Experimental%20Evaluation%20of%20Passive%20Optical%20Network%20Based%20Data%20Centre%20Architecture.pdf

Experimental Evaluation of Passive Optical Network Based Data Centre Architecture

Abstract

Similar works

Full text

Available Versions

White Rose Research Online