Towards a cloud enabler : from an optical network resource provisioning system to a generalized architecture for dynamic infrastructure services provisioning

This work was developed during a period where most of the optical management and provisioning system where manual and proprietary. This work contributed to the evolution of the state of the art of optical networks with new architectures and advanced virtual infrastructure services. The evolution of optical networks, and internet globally, have been very promising during the last decade. The impact of mobile technology, grid, cloud computing, HDTV, augmented reality and big data, among many others, have driven the evolution of optical networks towards current service technologies, mostly based on SDN (Software Defined Networking) architectures and NFV(Network Functions Virtualisation). Moreover, the convergence of IP/Optical networks and IT services, and the evolution of the internet and optical infrastructures, have generated novel service orchestrators and open source frameworks. In fact, technology has evolved that fast that none could foresee how important Internet is for our current lives. Said in other words, technology was forced to evolve in a way that network architectures became much more transparent, dynamic and flexible to the end users (applications, user interfaces or simple APIs). This Thesis exposes the work done on defining new architectures for Service Oriented Networks and the contribution to the state of the art. The research work is divided into three topics. It describes the evolution from a Network Resource Provisioning System to an advanced Service Plane, and ends with a new architecture that virtualized the optical infrastructure in order to provide coordinated, on-demand and dynamic services between the application and the network infrastructure layer, becoming an enabler for the new generation of cloud network infrastructures. The work done on defining a Network Resource Provisioning System established the first bases for future work on network infrastructure virtualization. The UCLP (User Light Path Provisioning) technology was the first attempt for Customer Empowered Networks and Articulated Private Networks. It empowered the users and brought virtualization and partitioning functionalities into the optical data plane, with new interfaces for dynamic service provisioning. The work done within the development of a new Service Plane allowed the provisioning of on-demand connectivity services from the application, and in a multi-domain and multi-technology scenario based on a virtual network infrastructure composed of resources from different infrastructure providers. This Service Plane facilitated the deployment of applications consuming large amounts of data under deterministic conditions, so allowing the networks behave as a Grid-class resource. It became the first on-demand provisioning system that at lower levels allowed the creation of one virtual domain composed from resources of different providers. The last research topic presents an architecture that consolidated the work done in virtualisation while enhancing the capabilities to upper layers, so fully integrating the optical network infrastructure into the cloud environment, and so providing an architecture that enabled cloud services by integrating the request of optical network and IT infrastructure services together at the same level. It set up a new trend into the research community and evolved towards the technology we use today based on SDN and NFV. Summing up, the work presented is focused on the provisioning of virtual infrastructures from the architectural point of view of optical networks and IT infrastructures, together with the design and definition of novel service layers. It means, architectures that enabled the creation of virtual infrastructures composed of optical networks and IT resources, isolated and provisioned on-demand and in advance with infrastructure re-planning functionalities, and a new set of interfaces to open up those services to applications or third parties.Aquesta tesi es va desenvolupar durant un període on la majoria de sistemes de gestió de xarxa òptica eren manuals i basats en sistemes propietaris. En aquest sentit, la feina presentada va contribuir a l'evolució de l'estat de l'art de les xarxes òptiques tant a nivell d’arquitectures com de provisió d’infraestructures virtuals. L'evolució de les xarxes òptiques, i d'Internet a nivell mundial, han estat molt prometedores durant l'última dècada. L'impacte de la tecnologia mòbil, la computació al núvol, la televisió d'alta definició, la realitat augmentada i el big data, entre molts altres, han impulsat l'evolució cap a xarxes d’altes prestacions amb nous serveis basats en SDN (Software Defined Networking) i NFV (Funcions de xarxa La virtualització). D'altra banda, la convergència de xarxes òptiques i els serveis IT, junt amb l'evolució d'Internet i de les infraestructures òptiques, han generat nous orquestradors de serveis i frameworks basats en codi obert. La tecnologia ha evolucionat a una velocitat on ningú podria haver predit la importància que Internet està tenint en el nostre dia a dia. Dit en altres paraules, la tecnologia es va veure obligada a evolucionar d'una manera on les arquitectures de xarxa es fessin més transparent, dinàmiques i flexibles vers als usuaris finals (aplicacions, interfícies d'usuari o APIs simples). Aquesta Tesi presenta noves arquitectures de xarxa òptica orientades a serveis. El treball de recerca es divideix en tres temes. Es presenta un sistema de virtualització i aprovisionament de recursos de xarxa i la seva evolució a un pla de servei avançat, per acabar presentant el disseny d’una nova arquitectura capaç de virtualitzar la infraestructura òptica i IT i proporcionar serveis de forma coordinada, i sota demanda, entre l'aplicació i la capa d'infraestructura de xarxa òptica. Tot esdevenint un facilitador per a la nova generació d'infraestructures de xarxa en el núvol. El treball realitzat en la definició del sistema de virtualització de recursos va establir les primeres bases sobre la virtualització de la infraestructura de xarxa òptica en el marc de les “Customer Empowered Networks” i “Articulated Private Networks”. Amb l’objectiu de virtualitzar el pla de dades òptic, i oferir noves interfícies per a la provisió de serveis dinàmics de xarxa. En quant al pla de serveis presentat, aquest va facilitat la provisió de serveis de connectivitat sota demanda per part de l'aplicació, tant en entorns multi-domini, com en entorns amb múltiples tecnologies. Aquest pla de servei, anomenat Harmony, va facilitar el desplegament de noves aplicacions que consumien grans quantitats de dades en condicions deterministes. En aquest sentit, va permetre que les xarxes es comportessin com un recurs Grid, i per tant, va esdevenir el primer sistema d'aprovisionament sota demanda que permetia la creació de dominis virtuals de xarxa composts a partir de recursos de diferents proveïdors. Finalment, es presenta l’evolució d’un pla de servei cap una arquitectura global que consolida el treball realitzat a nivell de convergència d’infraestructures (òptica + IT) i millora les capacitats de les capes superiors. Aquesta arquitectura va facilitar la plena integració de la infraestructura de xarxa òptica a l'entorn del núvol. En aquest sentit, aquest resultats van evolucionar cap a les tendències actuals de SDN i NFV. En resum, el treball presentat es centra en la provisió d'infraestructures virtuals des del punt de vista d’arquitectures de xarxa òptiques i les infraestructures IT, juntament amb el disseny i definició de nous serveis de xarxa avançats, tal i com ho va ser el servei de re-planificació dinàmicaPostprint (published version

Dimensionerings- en werkverdelingsalgoritmen voor lambda grids

Grids bestaan uit een verzameling reken- en opslagelementen die geografisch verspreid kunnen zijn, maar waarvan men de gezamenlijke capaciteit wenst te benutten. Daartoe dienen deze elementen verbonden te worden met een netwerk. Vermits veel wetenschappelijke applicaties gebruik maken van een Grid, en deze applicaties doorgaans grote hoeveelheden data verwerken, is het noodzakelijk om een netwerk te voorzien dat dergelijke grote datastromen op betrouwbare wijze kan transporteren. Optische transportnetwerken lenen zich hier uitstekend toe. Grids die gebruik maken van dergelijk netwerk noemt men lambda Grids. Deze thesis beschrijft een kader waarin het ontwerp en dimensionering van optische netwerken voor lambda Grids kunnen beschreven worden. Ook wordt besproken hoe werklast kan verdeeld worden op een Grid eens die gedimensioneerd is. Een groot deel van de resultaten werd bekomen door simulatie, waarbij gebruik gemaakt wordt van een eigen Grid simulatiepakket dat precies focust op netwerk- en Gridelementen. Het ontwerp van deze simulator, en de daarbijhorende implementatiekeuzes worden dan ook uitvoerig toegelicht in dit werk

Optimizing the delivery of multimedia over mobile networks

Mención Internacional en el título de doctorThe consumption of multimedia content is moving from a residential environment to mobile phones. Mobile data traffic, driven mostly by video demand, is increasing rapidly and wireless spectrum is becoming a more and more scarce resource. This makes it highly important to operate mobile networks efficiently. To tackle this, recent developments in anticipatory networking schemes make it possible to to predict the future capacity of mobile devices and optimize the allocation of the limited wireless resources. Further, optimizing Quality of Experience—smooth, quick, and high quality playback—is more difficult in the mobile setting, due to the highly dynamic nature of wireless links. A key requirement for achieving, both anticipatory networking schemes and QoE optimization, is estimating the available bandwidth of mobile devices. Ideally, this should be done quickly and with low overhead. In summary, we propose a series of improvements to the delivery of multimedia over mobile networks. We do so, be identifying inefficiencies in the interconnection of mobile operators with the servers hosting content, propose an algorithm to opportunistically create frequent capacity estimations suitable for use in resource optimization solutions and finally propose another algorithm able to estimate the bandwidth class of a device based on minimal traffic in order to identify the ideal streaming quality its connection may support before commencing playback. The main body of this thesis proposes two lightweight algorithms designed to provide bandwidth estimations under the high constraints of the mobile environment, such as and most notably the usually very limited traffic quota. To do so, we begin with providing a thorough overview of the communication path between a content server and a mobile device. We continue with analysing how accurate smartphone measurements can be and also go in depth identifying the various artifacts adding noise to the fidelity of on device measurements. Then, we first propose a novel lightweight measurement technique that can be used as a basis for advanced resource optimization algorithms to be run on mobile phones. Our main idea leverages an original packet dispersion based technique to estimate per user capacity. This allows passive measurements by just sampling the existing mobile traffic. Our technique is able to efficiently filter outliers introduced by mobile network schedulers and phone hardware. In order to asses and verify our measurement technique, we apply it to a diverse dataset generated by both extensive simulations and a week-long measurement campaign spanning two cities in two countries, different radio technologies, and covering all times of the day. The results demonstrate that our technique is effective even if it is provided only with a small fraction of the exchanged packets of a flow. The only requirement for the input data is that it should consist of a few consecutive packets that are gathered periodically. This makes the measurement algorithm a good candidate for inclusion in OS libraries to allow for advanced resource optimization and application-level traffic scheduling, based on current and predicted future user capacity. We proceed with another algorithm that takes advantage of the traffic generated by short-lived TCP connections, which form the majority of the mobile connections, to passively estimate the currently available bandwidth class. Our algorithm is able to extract useful information even if the TCP connection never exits the slow start phase. To the best of our knowledge, no other solution can operate with such constrained input. Our estimation method is able to achieve good precision despite artifacts introduced by the slow start behavior of TCP, mobile scheduler and phone hardware. We evaluate our solution against traces collected in 4 European countries. Furthermore, the small footprint of our algorithm allows its deployment on resource limited devices. Finally, in an attempt to face the rapid traffic increase, mobile application developers outsource their cloud infrastructure deployment and content delivery to cloud computing services and content delivery networks. Studying how these services, which we collectively denote Cloud Service Providers (CSPs), perform over Mobile Network Operators (MNOs) is crucial to understanding some of the performance limitations of today’s mobile apps. To that end, we perform the first empirical study of the complex dynamics between applications, MNOs and CSPs. First, we use real mobile app traffic traces that we gathered through a global crowdsourcing campaign to identify the most prevalent CSPs supporting today’s mobile Internet. Then, we investigate how well these services interconnect with major European MNOs at a topological level, and measure their performance over European MNO networks through a month-long measurement campaign on the MONROE mobile broadband testbed. We discover that the top 6 most prevalent CSPs are used by 85% of apps, and observe significant differences in their performance across different MNOs due to the nature of their services, peering relationships with MNOs, and deployment strategies. We also find that CSP performance in MNOs is affected by inflated path length, roaming, and presence of middleboxes, but not influenced by the choice of DNS resolver. We also observe that the choice of operator’s Point of Presence (PoP) may inflate by at least 20% the delay towards popular websites.This work has been supported by IMDEA Networks Institute.Programa Oficial de Doctorado en Ingeniería TelemáticaPresidente: Ahmed Elmokashfi.- Secretario: Rubén Cuevas Rumín.- Vocal: Paolo Din

Telecommunications Networks

This book guides readers through the basics of rapidly emerging networks to more advanced concepts and future expectations of Telecommunications Networks. It identifies and examines the most pressing research issues in Telecommunications and it contains chapters written by leading researchers, academics and industry professionals. Telecommunications Networks - Current Status and Future Trends covers surveys of recent publications that investigate key areas of interest such as: IMS, eTOM, 3G/4G, optimization problems, modeling, simulation, quality of service, etc. This book, that is suitable for both PhD and master students, is organized into six sections: New Generation Networks, Quality of Services, Sensor Networks, Telecommunications, Traffic Engineering and Routing

Quality of service in optical burst switching networks

Tese dout., Engenharia Electrónica e Computação, Universidade do Algarve, 2009Fundação para e Ciência e a Tecnologi

Entrega de conteúdos multimédia em over-the-top: caso de estudo das gravações automáticas

Doutoramento em Engenharia EletrotécnicaOver-The-Top (OTT) multimedia delivery is a very appealing approach for providing ubiquitous, exible, and globally accessible services capable of low-cost and unrestrained device targeting. In spite of its appeal, the underlying delivery architecture must be carefully planned and optimized to maintain a high Qualityof- Experience (QoE) and rational resource usage, especially when migrating from services running on managed networks with established quality guarantees. To address the lack of holistic research works on OTT multimedia delivery systems, this Thesis focuses on an end-to-end optimization challenge, considering a migration use-case of a popular Catch-up TV service from managed IP Television (IPTV) networks to OTT. A global study is conducted on the importance of Catch-up TV and its impact in today's society, demonstrating the growing popularity of this time-shift service, its relevance in the multimedia landscape, and tness as an OTT migration use-case. Catch-up TV consumption logs are obtained from a Pay-TV operator's live production IPTV service containing over 1 million subscribers to characterize demand and extract insights from service utilization at a scale and scope not yet addressed in the literature. This characterization is used to build demand forecasting models relying on machine learning techniques to enable static and dynamic optimization of OTT multimedia delivery solutions, which are able to produce accurate bandwidth and storage requirements' forecasts, and may be used to achieve considerable power and cost savings whilst maintaining a high QoE. A novel caching algorithm, Most Popularly Used (MPU), is proposed, implemented, and shown to outperform established caching algorithms in both simulation and experimental scenarios. The need for accurate QoE measurements in OTT scenarios supporting HTTP Adaptive Streaming (HAS) motivates the creation of a new QoE model capable of taking into account the impact of key HAS aspects. By addressing the complete content delivery pipeline in the envisioned content-aware OTT Content Delivery Network (CDN), this Thesis demonstrates that signi cant improvements are possible in next-generation multimedia delivery solutions.A entrega de conteúdos multimédia em Over-The-Top (OTT) e uma proposta atractiva para fornecer um serviço flexível e globalmente acessível, capaz de alcançar qualquer dispositivo, com uma promessa de baixos custos. Apesar das suas vantagens, e necessario um planeamento arquitectural detalhado e optimizado para manter níveis elevados de Qualidade de Experiência (QoE), em particular aquando da migração dos serviços suportados em redes geridas com garantias de qualidade pré-estabelecidas. Para colmatar a falta de trabalhos de investigação na área de sistemas de entrega de conteúdos multimédia em OTT, esta Tese foca-se na optimização destas soluções como um todo, partindo do caso de uso de migração de um serviço popular de Gravações Automáticas suportado em redes de Televisão sobre IP (IPTV) geridas, para um cenário de entrega em OTT. Um estudo global para aferir a importância das Gravações Automáticas revela a sua relevância no panorama de serviços multimédia e a sua adequação enquanto caso de uso de migração para cenários OTT. São obtidos registos de consumos de um serviço de produção de Gravações Automáticas, representando mais de 1 milhão de assinantes, para caracterizar e extrair informação de consumos numa escala e âmbito não contemplados ate a data na literatura. Esta caracterização e utilizada para construir modelos de previsão de carga, tirando partido de sistemas de machine learning, que permitem optimizações estáticas e dinâmicas dos sistemas de entrega de conteúdos em OTT através de previsões das necessidades de largura de banda e armazenamento, potenciando ganhos significativos em consumo energético e custos. Um novo mecanismo de caching, Most Popularly Used (MPU), demonstra um desempenho superior as soluções de referencia, quer em cenários de simulação quer experimentais. A necessidade de medição exacta da QoE em streaming adaptativo HTTP motiva a criaçao de um modelo capaz de endereçar aspectos específicos destas tecnologias adaptativas. Ao endereçar a cadeia completa de entrega através de uma arquitectura consciente dos seus conteúdos, esta Tese demonstra que são possíveis melhorias de desempenho muito significativas nas redes de entregas de conteúdos em OTT de próxima geração

Actas da 10ª Conferência sobre Redes de Computadores

Universidade do MinhoCCTCCentro AlgoritmiCisco SystemsIEEE Portugal Sectio

Dynamic Optical Networks for Data Centres and Media Production

This thesis explores all-optical networks for data centres, with a particular focus on network designs for live media production. A design for an all-optical data centre network is presented, with experimental verification of the feasibility of the network data plane. The design uses fast tunable (< 200 ns) lasers and coherent receivers across a passive optical star coupler core, forming a network capable of reaching over 1000 nodes. Experimental transmission of 25 Gb/s data across the network core, with combined wavelength switching and time division multiplexing (WS-TDM), is demonstrated. Enhancements to laser tuning time via current pre-emphasis are discussed, including experimental demonstration of fast wavelength switching (< 35 ns) of a single laser between all combinations of 96 wavelengths spaced at 50 GHz over a range wider than the optical C-band. Methods of increasing the overall network throughput by using a higher complexity modulation format are also described, along with designs for line codes to enable pulse amplitude modulation across the WS-TDM network core. The construction of an optical star coupler network core is investigated, by evaluating methods of constructing large star couplers from smaller optical coupler components. By using optical circuit switches to rearrange star coupler connectivity, the network can be partitioned, creating independent reserves of bandwidth and resulting in increased overall network throughput. Several topologies for constructing a star from optical couplers are compared, and algorithms for optimum construction methods are presented. All of the designs target strict criteria for the flexible and dynamic creation of multicast groups, which will enable future live media production workflows in data centres. The data throughput performance of the network designs is simulated under synthetic and practical media production traffic scenarios, showing improved throughput when reconfigurable star couplers are used compared to a single large star. An energy consumption evaluation shows reduced network power consumption compared to incumbent and other proposed data centre network technologies