Control Plane Hardware Design for Optical Packet Switched Data Centre Networks

Optical packet switching for intra-data centre networks is key to addressing traffic requirements. Photonic integration and wavelength division multiplexing (WDM) can overcome bandwidth limits in switching systems. A promising technology to build a nanosecond-reconfigurable photonic-integrated switch, compatible with WDM, is the semiconductor optical amplifier (SOA). SOAs are typically used as gating elements in a broadcast-and-select (B\&S) configuration, to build an optical crossbar switch. For larger-size switching, a three-stage Clos network, based on crossbar nodes, is a viable architecture. However, the design of the switch control plane, is one of the barriers to packet switching; it should run on packet timescales, which becomes increasingly challenging as line rates get higher. The scheduler, used for the allocation of switch paths, limits control clock speed. To this end, the research contribution was the design of highly parallel hardware schedulers for crossbar and Clos network switches. On a field-programmable gate array (FPGA), the minimum scheduler clock period achieved was 5.0~ns and 5.4~ns, for a 32-port crossbar and Clos switch, respectively. By using parallel path allocation modules, one per Clos node, a minimum clock period of 7.0~ns was achieved, for a 256-port switch. For scheduler application-specific integrated circuit (ASIC) synthesis, this reduces to 2.0~ns; a record result enabling scalable packet switching. Furthermore, the control plane was demonstrated experimentally. Moreover, a cycle-accurate network emulator was developed to evaluate switch performance. Results showed a switch saturation throughput at a traffic load 60\% of capacity, with sub-microsecond packet latency, for a 256-port Clos switch, outperforming state-of-the-art optical packet switches

Robustness of bus overlays in optical networks

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2002.Includes bibliographical references (p. 53-56).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Local area networks (LANs) nowadays use optical fiber as the medium of communication. This fiber is used to connect a collection of electro-optic nodes which form network clouds. A network cloud is a distribution network that connects several external nodes to the backbone, and often takes the form of a star or tree. Optical stars and trees have expensive and inefficient recovery schemes, and as a result, are not attractive options when designing networks. In order to solve this problem, we introduce a virtual topology that makes use of the robustness that is inherently present in a metropolitan area network (MAN) or wide area network (WAN) (long haul network). The virtual topology uses a folded bus scheme and includes some of the elements of the real topology (architecture). By optically bypassing some of the router/switch nodes in the physical architecture, the virtual topology yields better recovery performance and more efficient systems (with respect to cost related to bandwidth and recoverability). We present a bus overlay which uses simple access nodes and is robust to single failures. Our architecture allows the use of existing optical backbone infrastructure. We consider a linear folded bus architecture and introduce a T-shaped folded bus. Although buses are generally not able to recover from failures, we propose a loopback approach. Our approach allows optical bypass of some routers during normal operation, thus reducing the load on routers, but makes use of routers in case of failures. We analyze the behavior of our linear and T-shaped systems under average use and failure conditions. We show that certain simple characteristics of the traffic matrix give meaningful performance characterization. We show that our architecture provides solutions which limit loads on the router.by Ari Levon Libarikian.S.M

Optisen liityntäverkon ohjelmoitavien logiikkapiirien ohjelmoinnin suunnittelu

Tiedonsiirtoverkkojen rakenne elää muutoksen aikaa. Perinteisesti data-, puhelin- ja laajakaistaliikenne on siirretty erillisissä verkoissa, joiden samanaikainen ylläpitäminen tuottaa ylimääräisiä kustannuksia operaattoreille. Lisäksi uudet interaktiiviset palvelut vaativat toimiakseen sellaista palvelunlaatua, jota nykyiset tiedonsiirtoverkot eivät kykene tarjoamaan. Näiden epäkohtien johdosta tiedonsiirtoverkkojen kehityksessä tähdätään uusiin, entistä joustavampiin ja eri toiminnot yhdistäviin verkkoratkaisuihin, joissa tiedonsiirron palvelunlaadulla on entistä suurempi merkitys. Tällä hetkellä tiedonsiirtoverkkojen kokonaistiedonsiirtonopeuden kasvun pullonkaulatekijäksi on muodostunut kaupunkialueiden syöttöverkkojen monimutkaisuus, millä on suuri vaikutus myös tiedonsiirron palvelunlaatuun. TEKES-rahoitteisessa OAN-projektissa kehitetään uutta ja yksinkertaisempaa kaupunkialueen optista syöttöverkkoratkaisua, jolla alueen palvelunlaatua saataisiin parannettua. Tämä diplomityö on tehty osana OAN-projektia. Työssä kuvataan tiedonsiirtoverkkojen tämänhetkinen rakenne, tarkastellaan sen ongelmia ja tulevaisuuden kehityssuuntia, sekä esitetään pohdintojen pohjalta rakennettu malli tulevaisuuden tiedonsiirtoverkkojen rakenteelle. Työssä esitetään myös HDL-implementointiprosessiin kuuluvat työvaiheet sekä käydään läpi suunnitelma OAN-projektissa toteutettavan prototyypin ohjelmoitavien logiikkapiirien ohjelmaa varten

Error Behaviour In Optical Networks

Optical fibre communications are now widely used in many applications, including local area computer networks. I postulate that many future optical LANs will be required to operate with limited optical power budgets for a variety of reasons, including increased system complexity and link speed, low cost components and minimal increases in transmit power. Some developers will wish to run links with reduced power budget margins, and the received data in these systems will be more susceptible to errors than has been the case previously. The errors observed in optical systems are investigated using the particular case of Gigabit Ethernet on fibre as an example. Gigabit Ethernet is one of three popular optical local area interconnects which use 8B/10B line coding, along with Fibre Channel and Infiniband, and is widely deployed. This line encoding is also used by packet switched optical LANs currently under development. A probabilistic analysis follows the effects of a single channel error in a frame, through the line coding scheme and the MAC layer frame error detection mechanisms. Empirical data is used to enhance this original analysis, making it directly relevant to deployed systems. Experiments using Gigabit Ethernet on fibre with reduced power levels at the receiver to simulate the effect of limited power margins are described. It is found that channel bit error rate and packet loss rate have only a weakly deterministic relationship, due to interactions between a number of non-uniform error characteristics at various network sub-layers. Some data payloads suffer from high bit error rates and low packet loss rates, compared to others with lower bit error rates and yet higher packet losses. Experiments using real Internet traffic contribute to the development of a novel model linking packet loss, the payload damage rate, and channel bit error rate. The observed error behaviours at various points in the physical and data link layers are detailed. These include data-dependent channel errors; this error hot- spotting is in contrast to the failure modes observed in a copper-based system. It is also found that both multiple channel errors within a single code-group, and multiple error instances within a frame, occur more frequently than might be expected. The overall effects of these error characteristics on the ability of cyclic redundancy checks (CRCs) to detect errors, and on the performance of higher layers in the network, is considered. This dissertation contributes to the discussion of layer interactions, which may lead to un-foreseen performance issues at higher levels of the network stack, and extends it by considering the physical and data link layers for a common form of optical link. The increased risk of errors in future optical networks, and my findings for 8B/10B encoded optical links, demonstrate the need for a cross-layer understanding of error characteristics in such systems. The development of these new networks should take error performance into account in light of the particular requirements of the application in question.The UK Engineering and Physical Sciences Research Council and Marconi Corporation supported my work financially through an Industrial CASE studentship

Error Behaviour In Optical Networks

Optical fibre communications are now widely used in many applications, including local area computer networks. I postulate that many future optical LANs will be required to operate with limited optical power budgets for a variety of reasons, including increased system complexity and link speed, low cost components and minimal increases in transmit power. Some developers will wish to run links with reduced power budget margins, and the received data in these systems will be more susceptible to errors than has been the case previously. The errors observed in optical systems are investigated using the particular case of Gigabit Ethernet on fibre as an example. Gigabit Ethernet is one of three popular optical local area interconnects which use 8B/10B line coding, along with Fibre Channel and Infiniband, and is widely deployed. This line encoding is also used by packet switched optical LANs currently under development. A probabilistic analysis follows the effects of a single channel error in a frame, through the line coding scheme and the MAC layer frame error detection mechanisms. Empirical data is used to enhance this original analysis, making it directly relevant to deployed systems. Experiments using Gigabit Ethernet on fibre with reduced power levels at the receiver to simulate the effect of limited power margins are described. It is found that channel bit error rate and packet loss rate have only a weakly deterministic relationship, due to interactions between a number of non-uniform error characteristics at various network sub-layers. Some data payloads suffer from high bit error rates and low packet loss rates, compared to others with lower bit error rates and yet higher packet losses. Experiments using real Internet traffic contribute to the development of a novel model linking packet loss, the payload damage rate, and channel bit error rate. The observed error behaviours at various points in the physical and data link layers are detailed. These include data-dependent channel errors; this error hot- spotting is in contrast to the failure modes observed in a copper-based system. It is also found that both multiple channel errors within a single code-group, and multiple error instances within a frame, occur more frequently than might be expected. The overall effects of these error characteristics on the ability of cyclic redundancy checks (CRCs) to detect errors, and on the performance of higher layers in the network, is considered. This dissertation contributes to the discussion of layer interactions, which may lead to un-foreseen performance issues at higher levels of the network stack, and extends it by considering the physical and data link layers for a common form of optical link. The increased risk of errors in future optical networks, and my findings for 8B/10B encoded optical links, demonstrate the need for a cross-layer understanding of error characteristics in such systems. The development of these new networks should take error performance into account in light of the particular requirements of the application in question.The UK Engineering and Physical Sciences Research Council and Marconi Corporation supported my work financially through an Industrial CASE studentship

Broadband access networks using hybrid radio/fiber systems

Developing broadband access networks is one of the most urgent needs in the telecommunications world. The wireless systems provide an efficient solution to address the requirements for last mile connectivity of data, Internet and voice services Radio systems using millimetre-wave frequencies can supply home users with capacities in the order of 50-200 Mbit/s Such bit rates allow the transmission of broadband applications including digital TV, video-on-demand etc In order to provide the massive capacities that are required for the distribution of such broadband data between Central Station and Base Stations, optical fiber can be employed The enormous transmission bandwidth and low loss of the fiber ensure that high capacity microwave signals can be encoded on an optical carrier and successfully transmitted from a Central to Base Station. The goal of this project was to develop and test a radio over fiber communication system This involved investigating the generation of microwave optical signals for transmission in optical fiber, followed by an examination of the effect of fiber propagation on the microwave optical signals

Ethernet Networks for Real-Time Use in the ATLAS Experiment

Ethernet became today's de-facto standard technology for local area networks. Defined by the IEEE 802.3 and 802.1 working groups, the Ethernet standards cover technologies deployed at the first two layers of the OSI protocol stack. The architecture of modern Ethernet networks is based on switches. The switches are devices usually built using a store-and-forward concept. At the highest level, they can be seen as a collection of queues and mathematically modelled by means of queuing theory. However, the traffic profiles on modern Ethernet networks are rather different from those assumed in classical queuing theory. The standard recommendations for evaluating the performance of network devices define the values that should be measured but do not specify a way of reconciling these values with the internal architecture of the switches. The introduction of the 10 Gigabit Ethernet standard provided a direct gateway from the LAN to the WAN by the means of the WAN PHY. Certain aspects related to the actual use of WAN PHY technology were vaguely defined by the standard. The ATLAS experiment at CERN is scheduled to start operation at CERN in 2007. The communication infrastructure of the Trigger and Data Acquisition System will be built using Ethernet networks. The real-time operational needs impose a requirement for predictable performance on the network part. In view of the diversity of the architectures of Ethernet devices, testing and modelling is required in order to make sure the full system will operate predictably. This thesis focuses on the testing part of the problem and addresses issues in determining the performance for both LAN and WAN connections. The problem of reconciling results from measurements to architectural details of the switches will also be tackled. We developed a scalable traffic generator system based on commercial-off-the-shelf Gigabit Ethernet network interface cards. The generator was able to transmit traffic at the nominal Gigabit Ethernet line rate for all frame sizes specified in the Ethernet standard. The calculation of latency was performed with accuracy in the range of +/- 200 ns. We indicate how certain features of switch architectures may be identified through accurate throughput and latency values measured for specific traffic distributions. At this stage, we present a detailed analysis of Ethernet broadcast support in modern switches. We use a similar hands-on approach to address the problem of extending Ethernet networks over long distances. Based on the 1 Gbit/s traffic generator used in the LAN, we develop a methodology to characterise point-to-point connections over long distance networks. At higher speeds, a combination of commercial traffic generators and high-end servers is employed to determine the performance of the connection. We demonstrate that the new 10 Gigabit Ethernet technology can interoperate with the installed base of SONET/SDH equipment through a series of experiments on point-to-point circuits deployed over long-distance network infrastructure in a multi-operator domain. In this process, we provide a holistic view of the end-to-end performance of 10 Gigabit Ethernet WAN PHY connections through a sequence of measurements starting at the physical transmission layer and continuing up to the transport layer of the OSI protocol stack