Measurement-Driven Algorithm and System Design for Wireless and Datacenter Networks

The growing number of mobile devices and data-intensive applications pose unique challenges for wireless access networks as well as datacenter networks that enable modern cloud-based services. With the enormous increase in volume and complexity of traffic from applications such as video streaming and cloud computing, the interconnection networks have become a major performance bottleneck. In this thesis, we study algorithms and architectures spanning several layers of the networking protocol stack that enable and accelerate novel applications and that are easily deployable and scalable. The design of these algorithms and architectures is motivated by measurements and observations in real world or experimental testbeds. In the first part of this thesis, we address the challenge of wireless content delivery in crowded areas. We present the AMuSe system, whose objective is to enable scalable and adaptive WiFi multicast. AMuSe is based on accurate receiver feedback and incurs a small control overhead. This feedback information can be used by the multicast sender to optimize multicast service quality, e.g., by dynamically adjusting transmission bitrate. Specifically, we develop an algorithm for dynamic selection of a subset of the multicast receivers as feedback nodes which periodically send information about the channel quality to the multicast sender. Further, we describe the Multicast Dynamic Rate Adaptation (MuDRA) algorithm that utilizes AMuSe's feedback to optimally tune the physical layer multicast rate. MuDRA balances fast adaptation to channel conditions and stability, which is essential for multimedia applications. We implemented the AMuSe system on the ORBIT testbed and evaluated its performance in large groups with approximately 200 WiFi nodes. Our extensive experiments demonstrate that AMuSe can provide accurate feedback in a dense multicast environment. It outperforms several alternatives even in the case of external interference and changing network conditions. Further, our experimental evaluation of MuDRA on the ORBIT testbed shows that MuDRA outperforms other schemes and supports high throughput multicast flows to hundreds of nodes while meeting quality requirements. As an example application, MuDRA can support multiple high quality video streams, where 90% of the nodes report excellent or very good video quality. Next, we specifically focus on ensuring high Quality of Experience (QoE) for video streaming over WiFi multicast. We formulate the problem of joint adaptation of multicast transmission rate and video rate for ensuring high video QoE as a utility maximization problem and propose an online control algorithm called DYVR which is based on Lyapunov optimization techniques. We evaluated the performance of DYVR through analysis, simulations, and experiments using a testbed composed of Android devices and o the shelf APs. Our evaluation shows that DYVR can ensure high video rates while guaranteeing a low but acceptable number of segment losses, buffer underflows, and video rate switches. We leverage the lessons learnt from AMuSe for WiFi to address the performance issues with LTE evolved Multimedia Broadcast/Multicast Service (eMBMS). We present the Dynamic Monitoring (DyMo) system which provides low-overhead and real-time feedback about eMBMS performance. DyMo employs eMBMS for broadcasting instructions which indicate the reporting rates as a function of the observed Quality of Service (QoS) for each UE. This simple feedback mechanism collects very limited QoS reports which can be used for network optimization. We evaluated the performance of DyMo analytically and via simulations. DyMo infers the optimal eMBMS settings with extremely low overhead, while meeting strict QoS requirements under different UE mobility patterns and presence of network component failures. In the second part of the thesis, we study datacenter networks which are key enablers of the end-user applications such as video streaming and storage. Datacenter applications such as distributed file systems, one-to-many virtual machine migrations, and large-scale data processing involve bulk multicast flows. We propose a hardware and software system for enabling physical layer optical multicast in datacenter networks using passive optical splitters. We built a prototype and developed a simulation environment to evaluate the performance of the system for bulk multicasting. Our evaluation shows that the optical multicast architecture can achieve higher throughput and lower latency than IP multicast and peer-to-peer multicast schemes with lower switching energy consumption. Finally, we study the problem of congestion control in datacenter networks. Quantized Congestion Control (QCN), a switch-supported standard, utilizes direct multi-bit feedback from the network for hardware rate limiting. Although QCN has been shown to be fast-reacting and effective, being a Layer-2 technology limits its adoption in IP-routed Layer 3 datacenters. We address several design challenges to overcome QCN feedback's Layer- 2 limitation and use it to design window-based congestion control (QCN-CC) and load balancing (QCN-LB) schemes. Our extensive simulations, based on real world workloads, demonstrate the advantages of explicit, multi-bit congestion feedback, especially in a typical environment where intra-datacenter traffic with short Round Trip Times (RTT: tens of s) run in conjunction with web-facing traffic with long RTTs (tens of milliseconds)

Stochastische Analyse und lernbasierte Algorithmen zur Ressourcenbereitstellung in optischen Netzwerken

The unprecedented growth in Internet traffic has driven the innovations in provisioning of optical resources as per the need of bandwidth demands such that the resource utilization and spectrum efficiency could be maximized. With the advent of the next generation flexible optical transponders and switches, the flexible-grid-based elastic optical network (EON) is foreseen as an alternative to the widely deployed fixed-grid-based wavelength division multiplexing networks. At the same time, the flexible resource provisioning also raises new challenges for EONs. One such challenge is the spectrum fragmentation. As network traffic varies over time, spectrum gets fragmented due to the setting up and tearing down of non-uniform bandwidth requests over aligned (i.e., continuous) and adjacent (i.e., contiguous) spectrum slices, which leads to a non-optimal spectrum allocation, and generally results in higher blocking probability and lower spectrum utilization in EONs. To address this issue, the allocation and reallocation of optical resources are required to be modeled accurately, and managed efficiently and intelligently. The modeling of routing and spectrum allocation in EONs with the spectrum contiguity and spectrum continuity constraints is well-investigated, but existing models do not consider the fragmentation issue resulted by these constraints and non-uniform bandwidth demands. This thesis addresses this issue and considers both the constraints to computing exact blocking probabilities in EONs with and without spectrum conversion, and with spectrum reallocation (known as defragmentation) for the first time using the Markovian approach. As the exact network models are not scalable with respect to the network size and capacity, this thesis proposes load-independent and load-dependent approximate models to compute approximate blocking probabilities in EONs. Results show that the connection blocking due to fragmentation can be reduced by using a spectrum conversion or a defragmentation approach, but it can not be eliminated in a mesh network topology. This thesis also deals with the important network resource provisioning task in EONs. To this end, it first presents algorithmic solutions to efficiently allocate and reallocate spectrum resources using the fragmentation factor along spectral, time, and spatial dimensions. Furthermore, this thesis highlights the role of machine learning techniques in alleviating issues in static provisioning of optical resources, and presents two use-cases: handling time-varying traffic in optical data center networks, and reducing energy consumption and allocating spectrum proportionately to traffic classes in fiber-wireless networks.Die flexible Nutzung des Spektrums bringt in Elastischen Optischen Netze (EON) neue Herausforderungen mit sich, z.B., die Fragmentierung des Spektrums. Die Fragmentierung entsteht dadurch, dass die Netzwerkverkehrslast sich im Laufe der Zeit ändert und so wird das Spektrum aufgrund des Verbindungsaufbaus und -abbaus fragmentiert. Das für eine Verbindung notwendige Spektrum wird durch aufeinander folgende (kontinuierliche) und benachbarte (zusammenhängende) Spektrumsabschnitte (Slots) gebildet. Dies führt nach den zahlreichen Reservierungen und Freisetzungen des Spektrums zu einer nicht optimalen Zuordnung, die in einer höheren Blockierungs-wahrscheinlichkeit der neuen Verbindungsanfragen und einer geringeren Auslastung von EONs resultiert. Um dieses Problem zu lösen, müssen die Zuweisung und Neuzuordnung des Spektrums in EONs genau modelliert und effizient sowie intelligent verwaltet werden. Diese Arbeit beschäftigt sich mit dem Fragmentierungsproblem und berücksichtigt dabei die beiden Einschränkungen: Kontiguität und Kontinuität. Unter diesen Annahmen wurden analytische Modelle zur Berechnung einer exakten Blockierungswahrscheinlichkeit in EONs mit und ohne Spektrumskonvertierung erarbeitet. Außerdem umfasst diese Arbeit eine Analyse der Blockierungswahrscheinlichkeit im Falle einer Neuzuordnung des Sprektrums (Defragmentierung). Diese Blockierungsanalyse wird zum ersten Mal mit Hilfe der Markov-Modelle durchgeführt. Da die exakten analytischen Modelle hinsichtlich der Netzwerkgröße und -kapazität nicht skalierbar sind, werden in dieser Dissertation verkehrslastunabhängige und verkehrslastabhängige Approximationsmodelle vorgestellt. Diese Modelle bieten eine Näherung der Blockierungswahrscheinlichkeiten in EONs. Die Ergebnisse zeigen, dass die Blockierungswahrscheinlichkeit einer Verbindung aufgrund von einer Fragmentierung des Spektrums durch die Verwendung einer Spektrumkonvertierung oder eines Defragmentierungsverfahrens verringert werden kann. Eine effiziente Bereitstellung der optischen Netzwerkressourcen ist eine wichtige Aufgabe von EONs. Deswegen befasst sich diese Arbeit mit algorithmischen Lösungen, die Spektrumressource mithilfe des Fragmentierungsfaktors von Spektral-, Zeit- und räumlichen Dimension effizient zuweisen und neu zuordnen. Darüber hinaus wird die Rolle des maschinellen Lernens (ML) für eine verbesserte Bereitstellung der optischen Ressourcen untersucht und das ML basierte Verfahren mit der statischen Ressourcenzuweisung verglichen. Dabei werden zwei Anwendungsbeispiele vorgestellt und analysiert: der Umgang mit einer zeitveränderlichen Verkehrslast in optischen Rechenzentrumsnetzen, und eine Verringerung des Energieverbrauchs und die Zuweisung des Spektrums proportional zu Verkehrsklassen in kombinierten Glasfaser-Funknetzwerken

Communications

The communications sector of an economy comprises a range of technologies, physical media, and institutions/rules that facilitate the storage of information through means other than a society\u27s oral tradition and the transmission of that information over distances beyond the normal reach of human conversation. This chapter provides data on the historical evolution of a disparate range of industries and institutions contributing to the movement and storage of information in the United States over the past two centuries. These include the U.S. Postal Service, the newspaper industry, book publishing, the telegraph, wired and cellular telephone service, radio and television, and the Internet

Valuing Spectrum Allocations

Observing trends in which Wi-Fi and Bluetooth have become widely popular, some argue that unlicensed allocations hosting such wireless technologies are increasingly valuable and that administrative spectrum allocations should shift accordingly. We challenge that policy conclusion. A core issue is that the social value of a given spectrum allocation is widely assumed to equal the gains of the applications it is likely to host. This thinking is faulty, as vividly seen in what we deem the Broadcast TV Spectrum Valuation Fallacy – the idea that because wireless video, or broadcast network programs are popular, TV channels are efficiently defined. This approach has been appropriately rejected, in key instances, by spectrum regulators, but is similarly applied in other instances regarding unlicensed allocations. While traditional allocations have garnered widespread criticism for imposing rigid barriers tending to block innovation, and flexible-use spectrum access rights have gained favor, the regulatory methods used to allocate (or reallocate) bandwidth remain embedded in a “command and control” process. Reconfiguring spectrum usage to enable emerging wireless markets often requires lengthy, costly rule makings. The expense of this administrative overhead is generally omitted from spectrum allocation policy analysis. Yet, it constitutes an essential component of the consumer welfare analysis. We propose a more fulsome policy approach, one that includes not only the appropriate measures of marginal value and opportunity cost for rival allocations, but incorporates transaction costs. Instead of regulators attempting to guess how much bandwidth should be allocated to various types of licensed and unlicensed services – and imposing different rules within and across these allocations – a more generic approach is called for. By better enabling spontaneous adjustments to changing consumer demands and technological innovation, spectrum allocations can be more efficiently brought into their most valuable employments

AirSync: Enabling Distributed Multiuser MIMO with Full Spatial Multiplexing

The enormous success of advanced wireless devices is pushing the demand for higher wireless data rates. Denser spectrum reuse through the deployment of more access points per square mile has the potential to successfully meet the increasing demand for more bandwidth. In theory, the best approach to density increase is via distributed multiuser MIMO, where several access points are connected to a central server and operate as a large distributed multi-antenna access point, ensuring that all transmitted signal power serves the purpose of data transmission, rather than creating "interference." In practice, while enterprise networks offer a natural setup in which distributed MIMO might be possible, there are serious implementation difficulties, the primary one being the need to eliminate phase and timing offsets between the jointly coordinated access points. In this paper we propose AirSync, a novel scheme which provides not only time but also phase synchronization, thus enabling distributed MIMO with full spatial multiplexing gains. AirSync locks the phase of all access points using a common reference broadcasted over the air in conjunction with a Kalman filter which closely tracks the phase drift. We have implemented AirSync as a digital circuit in the FPGA of the WARP radio platform. Our experimental testbed, comprised of two access points and two clients, shows that AirSync is able to achieve phase synchronization within a few degrees, and allows the system to nearly achieve the theoretical optimal multiplexing gain. We also discuss MAC and higher layer aspects of a practical deployment. To the best of our knowledge, AirSync offers the first ever realization of the full multiuser MIMO gain, namely the ability to increase the number of wireless clients linearly with the number of jointly coordinated access points, without reducing the per client rate.Comment: Submitted to Transactions on Networkin

Valuing Spectrum Allocations

Observing trends in which Wi-Fi and Bluetooth have become widely popular, some argue that unlicensed allocations hosting such wireless technologies are increasingly valuable and that administrative spectrum allocations should shift accordingly. We challenge that policy conclusion. A core issue is that the social value of a given spectrum allocation is widely assumed to equal the gains of the applications it is likely to host. This thinking is faulty, as vividly seen in what we deem the Broadcast TV Spectrum Valuation Fallacy – the idea that because wireless video, or broadcast network programs are popular, TV channels are efficiently defined. This approach has been appropriately rejected, in key instances, by spectrum regulators, but is similarly applied in other instances regarding unlicensed allocations. While traditional allocations have garnered widespread criticism for imposing rigid barriers tending to block innovation, and flexible-use spectrum access rights have gained favor, the regulatory methods used to allocate (or reallocate) bandwidth remain embedded in a “command and control” process. Reconfiguring spectrum usage to enable emerging wireless markets often requires lengthy, costly rule makings. The expense of this administrative overhead is generally omitted from spectrum allocation policy analysis. Yet, it constitutes an essential component of the consumer welfare analysis. We propose a more fulsome policy approach, one that includes not only the appropriate measures of marginal value and opportunity cost for rival allocations, but incorporates transaction costs. Instead of regulators attempting to guess how much bandwidth should be allocated to various types of licensed and unlicensed services – and imposing different rules within and across these allocations – a more generic approach is called for. By better enabling spontaneous adjustments to changing consumer demands and technological innovation, spectrum allocations can be more efficiently brought into their most valuable employments

Wavelength reconfigurability for next generation optical access networks

Next generation optical access networks should not only increase the capacity but also be able to redistribute the capacity on the fly in order to manage larger variations in traffic patterns. Wavelength reconfigurability is the instrument to enable such capability of network-wide bandwidth redistribution since it allows dynamic sharing of both wavelengths and timeslots in WDM-TDM optical access networks. However, reconfigurability typically requires tunable lasers and tunable filters at the user side, resulting in cost-prohibitive optical network units (ONU). In this dissertation, I propose a novel concept named cyclic-linked flexibility to address the cost-prohibitive problem. By using the cyclic-linked flexibility, the ONU needs to switch only within a subset of two pre-planned wavelengths, however, the cyclic-linked structure of wavelengths allows free bandwidth to be shifted to any wavelength by a rearrangement process. Rearrangement algorithm are developed to demonstrate that the cyclic-linked flexibility performs close to the fully flexible network in terms of blocking probability, packet delay, and packet loss. Furthermore, the evaluation shows that the rearrangement process has a minimum impact to in-service ONUs. To realize the cyclic-linked flexibility, a family of four physical architectures is proposed. PRO-Access architecture is suitable for new deployments and disruptive upgrades in which the network reach is not longer than 20 km. WCL-Access architecture is suitable for metro-access merger with the reach up to 100 km. PSB-Access architecture is suitable to implement directly on power-splitter-based PON deployments, which allows coexistence with current technologies. The cyclically-linked protection architecture can be used with current and future PON standards when network protection is required