Transport Layer solution for bulk data transfers over Heterogeneous Long Fat Networks in Next Generation Networks

Aquesta tesi per compendi centra les seves contribucions en l'aprenentatge i innovació de les Xarxes de Nova Generació. És per això que es proposen diferents contribucions en diferents àmbits (Smart Cities, Smart Grids, Smart Campus, Smart Learning, Mitjana, eHealth, Indústria 4.0 entre d'altres) mitjançant l'aplicació i combinació de diferents disciplines (Internet of Things, Building Information Modeling, Cloud Storage, Ciberseguretat, Big Data, Internet de el Futur, Transformació Digital). Concretament, es detalla el monitoratge sostenible del confort a l'Smart Campus, la que potser es la meva aportació més representativa dins de la conceptualització de Xarxes de Nova Generació. Dins d'aquest innovador concepte de monitorització s'integren diferents disciplines, per poder oferir informació sobre el nivell de confort de les persones. Aquesta investigació demostra el llarg recorregut que hi ha en la transformació digital dels sectors tradicionals i les NGNs. Durant aquest llarg aprenentatge sobre les NGN a través de les diferents investigacions, es va poder observar una problemàtica que afectava de manera transversal als diferents camps d'aplicació de les NGNs i que aquesta podia tenir una afectació en aquests sectors. Aquesta problemàtica consisteix en el baix rendiment durant l'intercanvi de grans volums de dades sobre xarxes amb gran capacitat d'ample de banda i remotament separades geogràficament, conegudes com a xarxes elefant. Concretament, això afecta al cas d'ús d'intercanvi massiu de dades entre regions Cloud (Cloud Data Sharing use case). És per això que es va estudiar aquest cas d'ús i les diferents alternatives a nivell de protocols de transport,. S'estudien les diferents problemàtiques que pateixen els protocols i s'observa per què aquests no són capaços d'arribar a rendiments òptims. Deguda a aquesta situació, s'hipotetiza que la introducció de mecanismes que analitzen les mètriques de la xarxa i que exploten eficientment la capacitat de la mateixa milloren el rendiment dels protocols de transport sobre xarxes elefant heterogènies durant l'enviament massiu de dades. Primerament, es dissenya l’Adaptative and Aggressive Transport Protocol (AATP), un protocol de transport adaptatiu i eficient amb l'objectiu de millorar el rendiment sobre aquest tipus de xarxes elefant. El protocol AATP s'implementa i es prova en un simulador de xarxes i un testbed sota diferents situacions i condicions per la seva validació. Implementat i provat amb èxit el protocol AATP, es decideix millorar el propi protocol, Enhanced-AATP, sobre xarxes elefant heterogènies. Per això, es dissenya un mecanisme basat en el Jitter Ràtio que permet fer aquesta diferenciació. A més, per tal de millorar el comportament del protocol, s’adapta el seu sistema de fairness per al repartiment just dels recursos amb altres fluxos Enhanced-AATP. Aquesta evolució s'implementa en el simulador de xarxes i es realitzen una sèrie de proves. A l'acabar aquesta tesi, es conclou que les Xarxes de Nova Generació tenen molt recorregut i moltes coses a millorar causa de la transformació digital de la societat i de l'aparició de nova tecnologia disruptiva. A més, es confirma que la introducció de mecanismes específics en la concepció i operació dels protocols de transport millora el rendiment d'aquests sobre xarxes elefant heterogènies.Esta tesis por compendio centra sus contribuciones en el aprendizaje e innovación de las Redes de Nueva Generación. Es por ello que se proponen distintas contribuciones en diferentes ámbitos (Smart Cities, Smart Grids, Smart Campus, Smart Learning, Media, eHealth, Industria 4.0 entre otros) mediante la aplicación y combinación de diferentes disciplinas (Internet of Things, Building Information Modeling, Cloud Storage, Ciberseguridad, Big Data, Internet del Futuro, Transformación Digital). Concretamente, se detalla la monitorización sostenible del confort en el Smart Campus, la que se podría considerar mi aportación más representativa dentro de la conceptualización de Redes de Nueva Generación. Dentro de este innovador concepto de monitorización se integran diferentes disciplinas, para poder ofrecer información sobre el nivel de confort de las personas. Esta investigación demuestra el recorrido que existe en la transformación digital de los sectores tradicionales y las NGNs. Durante este largo aprendizaje sobre las NGN a través de las diferentes investigaciones, se pudo observar una problemática que afectaba de manera transversal a los diferentes campos de aplicación de las NGNs y que ésta podía tener una afectación en estos sectores. Esta problemática consiste en el bajo rendimiento durante el intercambio de grandes volúmenes de datos sobre redes con gran capacidad de ancho de banda y remotamente separadas geográficamente, conocidas como redes elefante, o Long Fat Networks (LFNs). Concretamente, esto afecta al caso de uso de intercambio de datos entre regiones Cloud (Cloud Data Data use case). Es por ello que se estudió este caso de uso y las diferentes alternativas a nivel de protocolos de transporte. Se estudian las diferentes problemáticas que sufren los protocolos y se observa por qué no son capaces de alcanzar rendimientos óptimos. Debida a esta situación, se hipotetiza que la introducción de mecanismos que analizan las métricas de la red y que explotan eficientemente la capacidad de la misma mejoran el rendimiento de los protocolos de transporte sobre redes elefante heterogéneas durante el envío masivo de datos. Primeramente, se diseña el Adaptative and Aggressive Transport Protocol (AATP), un protocolo de transporte adaptativo y eficiente con el objetivo maximizar el rendimiento sobre este tipo de redes elefante. El protocolo AATP se implementa y se prueba en un simulador de redes y un testbed bajo diferentes situaciones y condiciones para su validación. Implementado y probado con éxito el protocolo AATP, se decide mejorar el propio protocolo, Enhanced-AATP, sobre redes elefante heterogéneas. Además, con tal de mejorar el comportamiento del protocolo, se mejora su sistema de fairness para el reparto justo de los recursos con otros flujos Enhanced-AATP. Esta evolución se implementa en el simulador de redes y se realizan una serie de pruebas. Al finalizar esta tesis, se concluye que las Redes de Nueva Generación tienen mucho recorrido y muchas cosas a mejorar debido a la transformación digital de la sociedad y a la aparición de nueva tecnología disruptiva. Se confirma que la introducción de mecanismos específicos en la concepción y operación de los protocolos de transporte mejora el rendimiento de estos sobre redes elefante heterogéneas.This compendium thesis focuses its contributions on the learning and innovation of the New Generation Networks. That is why different contributions are proposed in different areas (Smart Cities, Smart Grids, Smart Campus, Smart Learning, Media, eHealth, Industry 4.0, among others) through the application and combination of different disciplines (Internet of Things, Building Information Modeling, Cloud Storage, Cybersecurity, Big Data, Future Internet, Digital Transformation). Specifically, the sustainable comfort monitoring in the Smart Campus is detailed, which can be considered my most representative contribution within the conceptualization of New Generation Networks. Within this innovative monitoring concept, different disciplines are integrated in order to offer information on people's comfort levels. . This research demonstrates the long journey that exists in the digital transformation of traditional sectors and New Generation Networks. During this long learning about the NGNs through the different investigations, it was possible to observe a problematic that affected the different application fields of the NGNs in a transversal way and that, depending on the service and its requirements, it could have a critical impact on any of these sectors. This issue consists of a low performance operation during the exchange of large volumes of data on networks with high bandwidth capacity and remotely geographically separated, also known as Elephant networks, or Long Fat Networks (LFNs). Specifically, this critically affects the Cloud Data Sharing use case. That is why this use case and the different alternatives at the transport protocol level were studied. For this reason, the performance and operation problems suffered by layer 4 protocols are studied and it is observed why these traditional protocols are not capable of achieving optimal performance. Due to this situation, it is hypothesized that the introduction of mechanisms that analyze network metrics and efficiently exploit network’s capacity meliorates the performance of Transport Layer protocols over Heterogeneous Long Fat Networks during bulk data transfers. First, the Adaptive and Aggressive Transport Protocol (AATP) is designed. An adaptive and efficient transport protocol with the aim of maximizing its performance over this type of elephant network.. The AATP protocol is implemented and tested in a network simulator and a testbed under different situations and conditions for its validation. Once the AATP protocol was designed, implemented and tested successfully, it was decided to improve the protocol itself, Enhanced-AATP, to improve its performance over heterogeneous elephant networks. In addition, in order to upgrade the behavior of the protocol, its fairness system is improved for the fair distribution of resources among other Enhanced-AATP flows. Finally, this evolution is implemented in the network simulator and a set of tests are carried out. At the end of this thesis, it is concluded that the New Generation Networks have a long way to go and many things to improve due to the digital transformation of society and the appearance of brand-new disruptive technology. Furthermore, it is confirmed that the introduction of specific mechanisms in the conception and operation of transport protocols improves their performance on Heterogeneous Long Fat Networks

Performance Optimization and Dynamics Control for Large-scale Data Transfer in Wide-area Networks

Transport control plays an important role in the performance of large-scale scientific and media streaming applications involving transfer of large data sets, media streaming, online computational steering, interactive visualization, and remote instrument control. In general, these applications have two distinctive classes of transport requirements: large-scale scientific applications require high bandwidths to move bulk data across wide-area networks, while media streaming applications require stable bandwidths to ensure smooth media playback. Unfortunately, the widely deployed Transmission Control Protocol is inadequate for such tasks due to its performance limitations. The purpose of this dissertation is to conduct rigorous analytical study of the design and performance of transport solutions, and develop an integrated transport solution in a systematical way to overcome the limitations of current transport methods. One of the primary challenges is to explore and compose a set of feasible route options with multiple constraints. Another challenge essentially arises from the randomness inherent in wide-area networks, particularly the Internet. This randomness must be explicitly accounted for to achieve both goodput maximization and stabilization over the constructed routes by suitably adjusting the source rate in response to both network and host dynamics.The superior and robust performance of the proposed transport solution is extensively evaluated in a simulated environment and further verified through real-life implementations and deployments over both Internet and dedicated connections under disparate network conditions in comparison with existing transport methods

An integrated transport solution to big data movement in high-performance networks

Extreme-scale e-Science applications in various domains such as earth science and high energy physics among multiple national institutions within the U.S. are generating colossal amounts of data, now frequently termed as “big data”. The big data must be stored, managed and moved to different geographical locations for distributed data processing and analysis. Such big data transfers require stable and high-speed network connections, which are not readily available in traditional shared IP networks such as the Internet. High-performance networking technologies and services featuring high bandwidth and advance reservation are being rapidly developed and deployed across the nation and around the globe to support such scientific applications. However, these networking technologies and services have not been fully utilized, mainly because: i) the use of these technologies and services often requires considerable domain knowledge and many application users are even not aware of their existence; and ii) the end-to-end data transfer performance largely depends on the transport protocol being used on the end hosts. The high-speed network path with reserved bandwidth in High-performance Networks has shifted the data transfer bottleneck from network segments in traditional IP networks to end hosts, which most existing transport protocols are not well suited to handle. In this dissertation, an integrated transport solution is proposed in support of data- and network-intensive applications in various science domains. This solution integrates three major components, i.e., i) transport-support workflow optimization, ii) transport profile generation, and iii) transport protocol design, into a unified framework. Firstly, a class of transport-support workflow optimization problems are formulated, where an appropriate set of resources and services are selected to compose the best transport-support workflow to meet user’s data transfer request in terms of various performance requirements. Secondly, a transport profiler named Transport Profile Generator (TPG) and its extended and accelerated version named FastProf are designed and implemented to characterize and enhance the end-to-end data transfer performance of a selected transport method over an established network path. Finally, several approaches based on rate and error threshold control are proposed to design a suite of data transfer protocols specifically tailored for big data transfer over dedicated connections. The proposed integrated transport solution is implemented and evaluated in: i) a local testbed with a single 10 Gb/s back-to-back connection and dual 10 Gb/s NIC-to-NIC connections; and ii) several wide-area networks with 10 Gb/s long-haul connections at collaborative sites including Oak Ridge National Laboratory, Argonne National Laboratory, and University of Chicago

Toward a versatile transport protocol

Les travaux présentés dans cette thèse ont pour but d'améliorer la couche transport de l'architecture réseau de l'OSI. La couche transport est de nos jour dominée par l'utilisation de TCP et son contrôle de congestion. Récemment de nouveaux mécanismes de contrôle de congestion ont été proposés. Parmi eux TCP Friendly Rate Control (TFRC) semble être le plus abouti. Cependant, tout comme TCP, ce mécanisme ne prend pas en compte ni les évolutions du réseau ni les nouveaux besoins des applications. La première contribution de cette thèse consiste en une spécialisation de TFRC afin d'obtenir un protocole de transport avisé de la Qualité de Service (QdS) spécialement défini pour des réseaux à QdS offrant une garantie de bande passante. Ce protocole combine un mécanisme de contrôle de congestion orienté QdS qui prend en compte la réservation de bande passante au niveau réseau, avec un service de fiabilité totale afin de proposer un service similaire à TCP. Le résultat de cette composition constitue le premier protocole de transport adapté à des réseau à garantie de bande passante. En même temps que cette expansion de service au niveau réseau, de nouvelles technologies ont été proposées et déployées au niveau physique. Ces nouvelles technologies sont caractérisées par leur affranchissement de support filaire et la mobilité des systèmes terminaux. De plus, elles sont généralement déployées sur des entités où la puissance de calcul et la disponibilité mémoire sont inférieures à celles des ordinateurs personnels. La deuxième contribution de cette thèse est la proposition d'une adaptation de TFRC à ces entités via la proposition d'une version allégée du récepteur. Cette version a été implémentée, évaluée quantitativement et ses nombreux avantages et contributions ont été démontrés par rapport à TFRC. Enfin, nous proposons une optimisation des implémentations actuelles de TFRC. Cette optimisation propose tout d'abord un nouvel algorithme pour l'initialisation du récepteur basé sur l'utilisation de l'algorithme de Newton. Nous proposons aussi l'introduction d'un outil nous permettant d'étudier plus en détails la manière dont est calculé le taux de perte du côté récepteur. ABSTRACT : This thesis presents three main contributions that aim to improve the transport layer of the current networking architecture. The transport layer is nowadays overruled by the use of TCP and its congestion control. Recently new congestion control mechanisms have been proposed. Among them, TCP Friendly Rate Control (TFRC) appears to be one of the most complete. Nevertheless this congestion control mechanism, as TCP, does not take into account either the evolution of the network in terms of Quality of Service and mobility or the evolution of the applications. The first contribution of this thesis is a specialisation TFRC congestion control to propose a QoS-aware Transport Protocol specifically designed to operate over QoS-enabled networks with bandwidth guarantee mechanisms. This protocol combines a QoS-aware congestion control, which takes into account networklevel bandwidth reservations, with full reliability in order mechanism to provide a transport service similar to TCP. As a result, we obtain the guaranteed throughput at the application level where TCP fails. This protocol is the first transport protocol compliant with bandwidth guaranteed networks. At the same time the set of network services expands, new technologies have been proposed and deployed at the physical layer. These new technologies are mainly characterised by communications done without wire constraint and the mobility of the end-systems. Furthermore, these technologies are usually deployed on entities where the CPU power and memory storage are limited. The second contribution of this thesis is therefore to propose an adaptation of TFRC to these entities. This is accomplished with the proposition of a new sender-based version of TFRC. This version has been implemented, evaluated and its numerous contributions and advantages compare to usual TFRC version have been demonstrated. Finally, we proposed an optimisation of actual implementations of TFRC. This optimisation first consists in the proposition of an algorithm based on a numerical analysis of the equation used in TFRC and the use of the Newton's algorithm. We furthermore give a first step, with the introduction of a new framework for TFRC, in order to better understand TFRC behaviour and to optimise the computation of the packet loss rate according to loss probability distribution

Methods and design issues for next generation network-aware applications

Networks are becoming an essential component of modern cyberinfrastructure and this work describes methods of designing distributed applications for high-speed networks to improve application scalability, performance and capabilities. As the amount of data generated by scientific applications continues to grow, to be able to handle and process it, applications should be designed to use parallel, distributed resources and high-speed networks. For scalable application design developers should move away from the current component-based approach and implement instead an integrated, non-layered architecture where applications can use specialized low-level interfaces. The main focus of this research is on interactive, collaborative visualization of large datasets. This work describes how a visualization application can be improved through using distributed resources and high-speed network links to interactively visualize tens of gigabytes of data and handle terabyte datasets while maintaining high quality. The application supports interactive frame rates, high resolution, collaborative visualization and sustains remote I/O bandwidths of several Gbps (up to 30 times faster than local I/O). Motivated by the distributed visualization application, this work also researches remote data access systems. Because wide-area networks may have a high latency, the remote I/O system uses an architecture that effectively hides latency. Five remote data access architectures are analyzed and the results show that an architecture that combines bulk and pipeline processing is the best solution for high-throughput remote data access. The resulting system, also supporting high-speed transport protocols and configurable remote operations, is up to 400 times faster than a comparable existing remote data access system. Transport protocols are compared to understand which protocol can best utilize high-speed network connections, concluding that a rate-based protocol is the best solution, being 8 times faster than standard TCP. An HD-based remote teaching application experiment is conducted, illustrating the potential of network-aware applications in a production environment. Future research areas are presented, with emphasis on network-aware optimization, execution and deployment scenarios

Integration of unidirectional technologies into wireless back-haul architecture

This thesis was submitted for the degree of Docter of Philosophy and awarded by Brunel University.Back-haul infrastructures of today's wireless operators must support the triple-play services demanded by the market or regulatory bodies. To cope with increasing capacity demand, the EU FP7 project CARMEN has developed a cost-effective heterogeneous multi-radio wireless back-haul architecture, which may also leverage the native multicast capabilities of broadcast technologies such as DVB-T to off-load high-bandwidth broadcast content delivery. However, the integration of such unidirectional technologies into a packet-switched architecture requires careful considerations. The contribution of this thesis is the investigation, design and evaluation of protocols and mechanisms facilitating the integration of such unidirectional technologies into the wireless back-haul architecture so that they can be configured and utilized by the spectrum and capacity optimization modules. This integration mainly concerns the control plane and, in particular, the aspects related to resource and capability descriptions, neighborhood, link and Multi Protocol Label Switching (MPLS) Label-Switched Path (LSP) monitoring, unicast and multicast LSP signalling as well as topology forming and maintenance. During the course of this study we have analyzed the problem space, proposed solutions to the resulting research questions and evaluated our approach. Our results show that the now Unidirectional Technology (UDT)-aware architecture can readily consider Unidirectional Technologies (UDTs) to distribute, for example, broadcast content

A Novel Approach for Implementing Worldwide Interoperability for Microwave Access for Video Surveillance

Video surveillance applications have experienced an increase in demand over the last decade. Surveillance systems can easily be found in places such as commercial offices, banks and traffic intersections, parks and recreational areas. Surveillance applications have the potential to be implemented on a WiMAX (Worldwide Interoperability for Microwave Access) network. Moreover, WiMAX devices have been used widely in the market and WiMAX-based video surveillance products have also been available. As a radio technology, WiMAX is a wireless broadband system that offers greater capacity than WiFi networks and wider coverage than cellular networks. The acceptance of WiMAX in the market, the availability of WiMAX products and its technology excellence, contribute to the possibility of implementing it for surveillance application. However, since WiMAX is designed to accommodate various applications with different quality of service (QoS) requirements, dedicated surveillance network implementation of WiMAX may not achieve optimum performance, as all Subscriber Stations (SSs) generate the same QoS requirements. In the medium access (MAC) layer, this thesis proposes a bandwidth allocation scheme that considers the QoS uniformity of the traffic sources. The proposed bandwidth allocation scheme comprises a simplified bandwidth allocation architecture, a packet-aware bandwidth request mechanism and packet-aware scheduling algorithms. The simplified architecture maximizes resources in the Base Station (BS), deactivates unnecessary services and minimizes the processing delay. The proposed bandwidth request mechanism reduces bandwidth grant and transmission delays. The proposed scheduling algorithms prioritize bandwidth granting access to a request that contains important packet(s). The proposed methods in the MAC layer are designed to be applied to existing devices in the market, without the necessity to change hardware. The transport protocol should be able to deliver video with sufficient quality while maintaining low delay connectivity. The proposed transport layer protocol is therefore designed to improve the existing user datagram protocol (UDP) performance by retransmitting packet loss selectively to increase the received video quality, and utilizing MAC support to achieve low delay connectivity. In order to overcome the limitations of the lower layers, this thesis employs a rateless code instead of transport layer redundancy in the application layer. Moreover, this thesis proposes post-decoding error concealment techniques as the last means to overcome packet loss. To evaluate the performances of the proposed methods, simulations are carried out using NS-2 simulator on Linux platform. The proposed methods are compared to existing works to measure their effectiveness. To facilitate the implementation of the transport layer protocols in practical scenarios, UDP packet modification is applied for each transport layer protocol.Indonesian Directorate General of Higher Education (DGHE/DIKTI

TCP RAPID: FROM THEORY TO PRACTICE

Delay and bandwidth-based alternatives to TCP congestion-control have been around for nearly three decades and have seen a recent surge in interest. However, such designs have faced significant resistance in being deployed on a wide-scale across the Internet—this has been mostly due to serious concerns about noise in delay measurements, pacing inter-packet gaps, and required changes to the standard TCP stack. With the advent of high-speed networking, some of these concerns become even more significant. This thesis considers Rapid, a recent proposal for ultra-high speed congestion control, which perhaps stretches each of these challenges to the greatest extent. Rapid adopts a framework of continuous fine-scale bandwidth probing and rate adapting. It requires finely-controlled inter-packet gaps, high-precision timestamping of received packets, and reliance on fine-scale changes in interpacket gaps. While simulation-based evaluations of Rapid show that it has outstanding performance gains along several important dimensions, these will not translate to the real-world unless the above challenges are addressed. This thesis identifies the key challenges TCP Rapid faces on real high-speed networks, including deployability in standard protocol stacks, precise inter-packet gap creation, achieving robust bandwidth estimation in the presence of noise, and a stability/adaptability trade-off. A Linux implementation of Rapid is designed and developed after carefully considering each of these challenges. The evaluations on a 10Gbps testbed confirm that the implementation can indeed achieve the claimed performance gains, and that it would not have been possible unless each of the above challenges was addressed.Doctor of Philosoph