Algorithms for advance bandwidth reservation in media production networks

Media production generally requires many geographically distributed actors (e.g., production houses, broadcasters, advertisers) to exchange huge amounts of raw video and audio data. Traditional distribution techniques, such as dedicated point-to-point optical links, are highly inefficient in terms of installation time and cost. To improve efficiency, shared media production networks that connect all involved actors over a large geographical area, are currently being deployed. The traffic in such networks is often predictable, as the timing and bandwidth requirements of data transfers are generally known hours or even days in advance. As such, the use of advance bandwidth reservation (AR) can greatly increase resource utilization and cost efficiency. In this paper, we propose an Integer Linear Programming formulation of the bandwidth scheduling problem, which takes into account the specific characteristics of media production networks, is presented. Two novel optimization algorithms based on this model are thoroughly evaluated and compared by means of in-depth simulation results

ETSI MANO network orchestration

In the modern era there is a big change in the way computer networks are conceived and the old version defined by hardware implementation is leaving space for a new one based upon software functions. This innovation is the Network Function Virtualization and indeed aims at easing the management of networks and reducing the costs of their maintenance by deploying Virtual Network Functions in standard general purpose servers. The transition to this solution involved the necessity to improve the performance of virtualization techniques and with the development of new solutions now it is possible to run multiple different functions in the same physical machine. This means that also the cloud computing benefits from this technology, having computing, storaging and networking resources all easily manageable and accessible due to their separation from the hardware underneath. Therefore it is important that while building this architecture the components are properly working and interacting together and that the virtualization techniques do not produce too much overhead compared to the performance of the hardware implementation. In this essay will be discussed the Network Function Virtualization and the Open Source MANO project, focusing on its descriptors architecture and functioning. To better demonstrate how to create network topologies through these files, some examples are created and analyzed

An SDN QoE Monitoring Framework for VoIP and video applications

Τα τελευταία χρόνια έχει σημειωθεί ραγδαία άνοδος του κλάδου των κινητών επικοινωνιών, αφού η χρήση των κινητών συσκευών εξαπλώνεται με ταχύτατους ρυθμούς και αναμένεται να συνεχίσει τη διείσδυσή της στην καθημερινότητα των καταναλωτών. Το γεγονός αυτό, σε συνδυασμό με τους περιορισμούς που θέτει η τρέχουσα δομή των δικτύων επικοινωνιών, καθιστά αναγκαία την ανάπτυξη νέων δικτύων με αυξημένες δυνατότητες, ώστε να είναι δυνατή η εξυπηρέτηση των χρηστών με την καλύτερη δυνατή ποιότητα εμπειρίας και ταυτόχρονα τη βέλτιστη αξιοποίηση των πόρων του δικτύου. Μία νέα δικτυακή προσέγγιση αποτελεί η δικτύωση βασισμένη στο λογισμικό (Software Defined Networking - SDN), η οποία αφαιρεί τον έλεγχο από τις συσκευές προώθησης του δικτύου, και οι αποφάσεις λαμβάνονται σε κεντρικό σημείο. Η ποιότητα υπηρεσίας που αντιλαμβάνεται ο χρήστης, ή αλλιώς ποιότητα εμπειρίας, κρίνεται ζήτημα υψηλής σημασίας στα δίκτυα SDN. Η παρούσα διπλωματική εργασία έχει ως στόχο την παρουσίαση της τεχνολογίας SDN, την επισκόπηση της υπάρχουσας έρευνας στο πεδίο της ποιότητας εμπειρίας σε SDN δίκτυα και στη συνέχεια την ανάπτυξη μίας SDN εφαρμογής η οποία παρακολουθεί και διατηρεί την ποιότητας εμπειρίας σε υψηλά επίπεδα για εφαρμογές VoIP και video. Πιο συγκεκριμένα, η εφαρμογή SQMF (SDN QoE Monitoring Framework) παρακολουθεί περιοδικά στο μονοπάτι μετάδοσης των πακέτων διάφορες παραμέτρους του δικτύου, με βάση τις οποίες υπολογίζει την ποιότητα εμπειρίας. Εάν διαπιστωθεί ότι το αποτέλεσμα είναι μικρότερο από ένα προσδιορισμένο κατώφλι, η εφαρμογή αλλάζει το μονοπάτι μετάδοσης, και έτσι η ποιότητα εμπειρίας ανακάμπτει. Η δομή της παρούσας διπλωματικής εργασίας είναι η εξής: Στο κεφάλαιο 1 παρουσιάζεται η σημερινή εικόνα των δικτύων επικοινωνιών και οι προβλέψεις για τη μελλοντική εικόνα, καθώς και οι προκλήσεις στις οποίες τα σημερινά δίκτυα δε θα μπορούν να αντεπεξέλθουν. Στη συνέχεια στο κεφάλαιο 2 περιγράφεται αναλυτικά η τεχνολογία SDN ως προς την αρχιτεκτονική, το κύριο πρωτόκολλο που χρησιμοποιεί, τα σενάρια χρήσης της, την προτυποποίηση, τα πλεονεκτήματα και τα μειονεκτήματά της. Το κεφάλαιο 3 εισάγει την έννοια της ποιότητας εμπειρίας του χρήστη και παραθέτει ευρέως γνωστά μοντέλα υπολογισμού της για διάφορους τύπους εφαρμογών, που χρησιμοποιούνται στην παρούσα εργασία. Σχετικές υπάρχουσες μελέτες στο πεδίο της ποιότητας εμπειρίας σε δίκτυα SDN αλλά και συγκριτικός πίνακας μπορούν να βρεθούν στο κεφάλαιο 4. Τα επόμενα κεφάλαια αφορούν στην εφαρμογή SQMF που υλοποιήθηκε στα πλαίσια της παρούσας διπλωματικής εργασίας: το κεφάλαιο 5 περιγράφει αναλυτικά όλα τα προαπαιτούμενα εργαλεία και οδηγίες για την ανάπτυξη του SQMF, ενώ το κεφάλαιο 6 παρουσιάζει παραδείγματα όπου η ποιότητα εμπειρίας ενός δικτύου μπορεί να υποστεί μείωση. Τέλος, το κεφάλαιο 7 αναλύει σε βάθος τις σχεδιαστικές προδιαγραφές, τη λογική και τον κώδικα του SQMF και παρέχει επίδειξη της λειτουργίας του και αξιολόγησή του, ενώ το κεφάλαιο 8 συνοψίζει επιγραμματικά τα συμπεράσματα της παρούσας εργασίας και ανοιχτά θέματα για μελλοντική έρευνα.Lately, there has been a rapid rise of the mobile communications industry, since the use of mobile devices is spreading at a fast pace and is expected to continue its penetration into the daily routine of consumers. This fact, combined with the limitations of the current communications networks’ structure, necessitates the development of new networks with increased capabilities, so that users can be served with the best possible quality of service and at the same time with the optimal network resources utilization. A new networking approach is Software Defined Networking (SDN) which decouples the control from the data plane, transforming the network elements to simple forwarding devices and making decisions centrally. The quality of service perceived by the user, or quality of experience (QoE), is considered to be a matter of great importance in software defined networks. This diploma thesis aims at presenting SDN technology, reviewing existing research in the field of QoE on SDN networks and then developing an SDN application that monitors and preserves the QoE for VoIP and video applications. More specifically, the developed SDN QoE Monitoring Framework (SQMF) periodically monitors various network parameters on the VoIP/video packets transmission path, based on which it calculates the QoE. If it is found that the result is less than a predefined threshold, the framework changes the transmission path, and thus the QoE recovers. The structure of this diploma thesis is the following: Chapter 1 presents the current state of communications networks and predictions for the future state, as well as the challenges that current networks will not be able to cope with. Chapter 2 then describes in detail the SDN technology in terms of architecture, main control-data plane communication protocol, use cases, standardization, advantages and disadvantages. Chapter 3 introduces the concept of QoE and lists well-known QoE estimation models for various applications types, some of which were used in this thesis. Relevant existing studies in the field of QoE on SDN networks as well as a comparative table can be found in chapter 4. The following chapters concern the framework implemented in the context of this diploma thesis: Chapter 5 describes in detail all the required tools and instructions for the development of SQMF, while Chapter 6 presents examples where the QoE in a network can face degradation. Finally, Chapter 7 analyzes in depth SQMF's design principles, logic and code files, provides a demonstration of its operation and evaluates it, whereas Chapter 8 briefly summarizes the conclusions and of this thesis and future work points

Orchestration of distributed ingestion and processing of IoT data for fog platforms

In recent years there has been an extraordinary growth of the Internet of Things (IoT) and its protocols. The increasing diffusion of electronic devices with identification, computing and communication capabilities is laying ground for the emergence of a highly distributed service and networking environment. The above mentioned situation implies that there is an increasing demand for advanced IoT data management and processing platforms. Such platforms require support for multiple protocols at the edge for extended connectivity with the objects, but also need to exhibit uniform internal data organization and advanced data processing capabilities to fulfill the demands of the application and services that consume IoT data. One of the initial approaches to address this demand is the integration between IoT and the Cloud computing paradigm. There are many benefits of integrating IoT with Cloud computing. The IoT generates massive amounts of data, and Cloud computing provides a pathway for that data to travel to its destination. But today’s Cloud computing models do not quite fit for the volume, variety, and velocity of data that the IoT generates. Among the new technologies emerging around the Internet of Things to provide a new whole scenario, the Fog Computing paradigm has become the most relevant. Fog computing was introduced a few years ago in response to challenges posed by many IoT applications, including requirements such as very low latency, real-time operation, large geo-distribution, and mobility. Also this low latency, geo-distributed and mobility environments are covered by the network architecture MEC (Mobile Edge Computing) that provides an IT service environment and Cloud-computing capabilities at the edge of the mobile network, within the Radio Access Network (RAN) and in close proximity to mobile subscribers. Fog computing addresses use cases with requirements far beyond Cloud-only solution capabilities. The interplay between Cloud and Fog computing is crucial for the evolution of the so-called IoT, but the reach and specification of such interplay is an open problem. This thesis aims to find the right techniques and design decisions to build a scalable distributed system for the IoT under the Fog Computing paradigm to ingest and process data. The final goal is to explore the trade-offs and challenges in the design of a solution from Edge to Cloud to address opportunities that current and future technologies will bring in an integrated way. This thesis describes an architectural approach that addresses some of the technical challenges behind the convergence between IoT, Cloud and Fog with special focus on bridging the gap between Cloud and Fog. To that end, new models and techniques are introduced in order to explore solutions for IoT environments. This thesis contributes to the architectural proposals for IoT ingestion and data processing by 1) proposing the characterization of a platform for hosting IoT workloads in the Cloud providing multi-tenant data stream processing capabilities, the interfaces over an advanced data-centric technology, including the building of a state-of-the-art infrastructure to evaluate the performance and to validate the proposed solution. 2) studying an architectural approach following the Fog paradigm that addresses some of the technical challenges found in the first contribution. The idea is to study an extension of the model that addresses some of the central challenges behind the converge of Fog and IoT. 3) Design a distributed and scalable platform to perform IoT operations in a moving data environment. The idea after study data processing in Cloud, and after study the convenience of the Fog paradigm to solve the IoT close to the Edge challenges, is to define the protocols, the interfaces and the data management to solve the ingestion and processing of data in a distributed and orchestrated manner for the Fog Computing paradigm for IoT in a moving data environment.En els últims anys hi ha hagut un gran creixement del Internet of Things (IoT) i els seus protocols. La creixent difusió de dispositius electrònics amb capacitats d'identificació, computació i comunicació esta establint les bases de l’aparició de serveis altament distribuïts i del seu entorn de xarxa. L’esmentada situació implica que hi ha una creixent demanda de plataformes de processament i gestió avançada de dades per IoT. Aquestes plataformes requereixen suport per a múltiples protocols al Edge per connectivitat amb el objectes, però també necessiten d’una organització de dades interna i capacitats avançades de processament de dades per satisfer les demandes de les aplicacions i els serveis que consumeixen dades IoT. Una de les aproximacions inicials per abordar aquesta demanda és la integració entre IoT i el paradigma del Cloud computing. Hi ha molts avantatges d'integrar IoT amb el Cloud. IoT genera quantitats massives de dades i el Cloud proporciona una via perquè aquestes dades viatgin a la seva destinació. Però els models actuals del Cloud no s'ajusten del tot al volum, varietat i velocitat de les dades que genera l'IoT. Entre les noves tecnologies que sorgeixen al voltant del IoT per proporcionar un escenari nou, el paradigma del Fog Computing s'ha convertit en la més rellevant. Fog Computing es va introduir fa uns anys com a resposta als desafiaments que plantegen moltes aplicacions IoT, incloent requisits com baixa latència, operacions en temps real, distribució geogràfica extensa i mobilitat. També aquest entorn està cobert per l'arquitectura de xarxa MEC (Mobile Edge Computing) que proporciona serveis de TI i capacitats Cloud al edge per la xarxa mòbil dins la Radio Access Network (RAN) i a prop dels subscriptors mòbils. El Fog aborda casos d?us amb requisits que van més enllà de les capacitats de solucions només Cloud. La interacció entre Cloud i Fog és crucial per a l'evolució de l'anomenat IoT, però l'abast i especificació d'aquesta interacció és un problema obert. Aquesta tesi té com objectiu trobar les decisions de disseny i les tècniques adequades per construir un sistema distribuït escalable per IoT sota el paradigma del Fog Computing per a ingerir i processar dades. L'objectiu final és explorar els avantatges/desavantatges i els desafiaments en el disseny d'una solució des del Edge al Cloud per abordar les oportunitats que les tecnologies actuals i futures portaran d'una manera integrada. Aquesta tesi descriu un enfocament arquitectònic que aborda alguns dels reptes tècnics que hi ha darrere de la convergència entre IoT, Cloud i Fog amb especial atenció a reduir la bretxa entre el Cloud i el Fog. Amb aquesta finalitat, s'introdueixen nous models i tècniques per explorar solucions per entorns IoT. Aquesta tesi contribueix a les propostes arquitectòniques per a la ingesta i el processament de dades IoT mitjançant 1) proposant la caracterització d'una plataforma per a l'allotjament de workloads IoT en el Cloud que proporcioni capacitats de processament de flux de dades multi-tenant, les interfícies a través d'una tecnologia centrada en dades incloent la construcció d'una infraestructura avançada per avaluar el rendiment i validar la solució proposada. 2) estudiar un enfocament arquitectònic seguint el paradigma Fog que aborda alguns dels reptes tècnics que es troben en la primera contribució. La idea és estudiar una extensió del model que abordi alguns dels reptes centrals que hi ha darrere de la convergència de Fog i IoT. 3) Dissenyar una plataforma distribuïda i escalable per a realitzar operacions IoT en un entorn de dades en moviment. La idea després d'estudiar el processament de dades a Cloud, i després d'estudiar la conveniència del paradigma Fog per resoldre el IoT prop dels desafiaments Edge, és definir els protocols, les interfícies i la gestió de dades per resoldre la ingestió i processament de dades en un distribuït i orquestrat per al paradigma Fog Computing per a l'IoT en un entorn de dades en moviment

Network Function Virtualization: state-of-the-art and research challenges

Network Function Virtualization (NFV) has drawn significant attention from both industry and academia as an important shift in telecommunication service provisioning. By decoupling Network Functions (NFs) from the physical devices on which they run, NFV has the potential to lead to significant reductions in Operating Expenses (OPEX) and Capital Expenses (CAPEX) and facilitate the deployment of new services with increased agility and faster time-to-value. The NFV paradigm is still in its infancy and there is a large spectrum of opportunities for the research community to develop new architectures, systems and applications, and to evaluate alternatives and trade-offs in developing technologies for its successful deployment. In this paper, after discussing NFV and its relationship with complementary fields of Software Defined Networking (SDN) and cloud computing, we survey the state-of-the-art in NFV, and identify promising research directions in this area. We also overview key NFV projects, standardization efforts, early implementations, use cases and commercial products.Peer ReviewedPostprint (author's final draft

Virtual network function development for NG-PON Access Network Architecture

Dissertação de mestrado em Engenharia de Redes e Serviços TelemáticosThe access to Internet services on a large scale, high throughput and low latency has grown at a very high pace over time, with a growing demand for media content and applications increasingly oriented towards data consumption. This fact about the use of data at the edge of the network requires the Central Offices (CO) of telecommunication providers, to be pre pared to absorb these demands. COs generally offer data from various access methods, such as Passive Optical Network (PON) technologies, mobile networks, copper wired and oth ers. For each of these technologies there may be different manufacturers that support only their respective hardware and software solutions, although they all share different network resources and have management, configuration and monitoring tools (Fault, Configuration, Accounting, Performance, and Security management - FCAPS) similar, but being distinct and isolated from each other, which produces huge investment in Capital Expenditure (CAPEX) and Operational Expenditure (OPEX) and can cause barriers to innovation. Such panora mas forced the development of more flexible, scalable solutions that share platforms and net work architectures that can meet this need and enable the evolution of networks. It is then proposed the architecture of Software-Defined Network (SDN) which has in its proposal to abstract the control plane from the data plane, in addition to the virtualization of several Net work Function Virtualization (NFV). The SDN architecture allows APIs and protocols such as Openflow, NETCONF / YANG, RESTCONF, gRPC and others to be used so that there is communication between the various hardware and software elements that compose the net work and consume network resources, such as services AAA, DHCP, routing, orchestration, management or various applications that may exist in this context. This work then aims at the development of a virtualized network function, namely a VNF in the context of network security to be integrated as a component of an architecture guided by the SDN paradigm applied to broadband networks, and also adherent to the architecture OB-BAA promoted by the Broadband Forum. Such OB-BAA architecture fits into the initia tive to modernize the Information Technology (IT) components of broadband networks, more specifically the Central Offices. With such development, it was intended to explore the con cepts of network security, such as the IEEE 802.1X protocol applied in NG-PON networks for authentication and authorization of new network equipment. To achieve this goal, the development of the applications was based on the Golang language combined with gRPC programmable interfaces for communication between the various elements of the architec ture. Network emulators were initially used, and then the components were ”containerized” and inserted in the Docker and Kubernetes virtualization frameworks. Finally, performance metrics were analyzed in the usage tests, namely computational resource usage metrics (CPU, memory and network I/O), in addition to the execution time of several processes performed by the developed applications.O acesso aos serviços de Internet em larga escala, alto débito e baixa latência têm crescido em um ritmo bastante elevado ao longo dos tempos, com uma demanda crescente por conteúdos de media e aplicações cada vez mais orientadas ao consumo de dados. Tal fato acerca da uti lização de dados na periferia da rede, obriga a que os Central Offices (CO) dos provedores de telecomunicações estejam preparados para absorver estas demandas. Os CO geralmente re cebem dados de diversos métodos de acesso, como tecnologias Passive Optical Network (PON), redes móveis, cabladas em cobre, entre outros. Para cada uma destas tecnologias pode haver diferentes fabricantes que suportam somente suas respetivas soluções de hardware e software, apesar de todas compartilharem diversos recursos de rede e possuírem ferramentas de gestão, configuração e monitoração (Fault-management, Configuration, Accounting, Performance e Segurança - FCAPS) similares, mas serem distintas e isoladas entre si, o que se traduz em um enorme investimento em Capital Expenditure (CAPEX) e Operational Expenditure (OPEX) e pode causar barreiras à inovação. Tais panoramas forçaram o desenvolvimento de soluções mais flexíveis, escaláveis e que compartilhem plataformas e arquiteturas de redes que pos sam suprir tal necessidade e possibilitar a evolução das redes. Propõe-se então a arquitetura de redes definidas por software (Software-Defined Network - SDN) que tem em sua proposta abstrair o plano de controle do plano de dados, além da virtualização de diversas funções de rede (Network Function Virtualization - NFV). A arquitetura SDN possibilita que API’s e pro tocolos como Openflow, NETCONF/YANG, RESTCONF, gRPC e outros, sejam utilizados para que haja comunicação entre os diversos elementos de hardware e software que estejam a compor a rede e a consumir recursos de redes, como serviços de AAA, DHCP, roteamento, orquestração, gestão ou diversas outras aplicações que possam existir neste contexto. Este trabalho visa então o desenvolvimento de uma função de rede virtualizada nomeada mente uma (Virtual Network Function - VNF) no âmbito de segurança de redes a ser integrada como um componente de uma arquitetura orientada pelo paradigma de SDN aplicado a re des de banda larga, e aderente também à arquitetura OB-BAA promovida pelo Broadband Fo rum. Tal arquitetura OB-BAA se enquadra na iniciativa de modernização dos componentes de Tecnologia da Informação (TI) das redes de banda larga, mais especificamente dos Cen tral Offices. Com tal desenvolvimento pretende-se explorar conceitos de segurança de redes, como o protocolo IEEE 802.1X aplicado em redes NG-PON para autenticação e autorização de novos equipamentos de rede. Para atingir tal objetivo, utilizou-se desenvolvimento de aplicações baseadas na linguagem Golang aliado com interfaces programáveis gRPC para comunicação entre os diversos elementos da arquitetura. Para emular tais componentes, utilizou-se inicialmente emuladores de rede, e em um segundo momento os componentes foram ”containerizados” e inseridos nos frameworks de virtualização Docker e Kubernetes.Por fim, foram analisadas métricas de desempenho nos testes executados, nomeadamente métricas de utilização de recursos computacionais (CPU, memória e tráfego de rede), além do tempo de execução de diversos processos desempenhados pelas aplicações desenvolvidas

AUTOMATED NETWORK SECURITY WITH EXCEPTIONS USING SDN

Campus networks have recently experienced a proliferation of devices ranging from personal use devices (e.g. smartphones, laptops, tablets), to special-purpose network equipment (e.g. firewalls, network address translation boxes, network caches, load balancers, virtual private network servers, and authentication servers), as well as special-purpose systems (badge readers, IP phones, cameras, location trackers, etc.). To establish directives and regulations regarding the ways in which these heterogeneous systems are allowed to interact with each other and the network infrastructure, organizations typically appoint policy writing committees (PWCs) to create acceptable use policy (AUP) documents describing the rules and behavioral guidelines that all campus network interactions must abide by. While users are the audience for AUP documents produced by an organization\u27s PWC, network administrators are the responsible party enforcing the contents of such policies using low-level CLI instructions and configuration files that are typically difficult to understand and are almost impossible to show that they do, in fact, enforce the AUPs. In other words, mapping the contents of imprecise unstructured sentences into technical configurations is a challenging task that relies on the interpretation and expertise of the network operator carrying out the policy enforcement. Moreover, there are multiple places where policy enforcement can take place. For example, policies governing servers (e.g., web, mail, and file servers) are often encoded into the server\u27s configuration files. However, from a security perspective, conflating policy enforcement with server configuration is a dangerous practice because minor server misconfigurations could open up avenues for security exploits. On the other hand, policies that are enforced in the network tend to rarely change over time and are often based on one-size-fits-all policies that can severely limit the fast-paced dynamics of emerging research workflows found in campus networks. This dissertation addresses the above problems by leveraging recent advances in Software-Defined Networking (SDN) to support systems that enable novel in-network approaches developed to support an organization\u27s network security policies. Namely, we introduce PoLanCO, a human-readable yet technically-precise policy language that serves as a middle-ground between the imprecise statements found in AUPs and the technical low-level mechanisms used to implement them. Real-world examples show that PoLanCO is capable of implementing a wide range of policies found in campus networks. In addition, we also present the concept of Network Security Caps, an enforcement layer that separates server/device functionality from policy enforcement. A Network Security Cap intercepts packets coming from, and going to, servers and ensures policy compliance before allowing network devices to process packets using the traditional forwarding mechanisms. Lastly, we propose the on-demand security exceptions model to cope with the dynamics of emerging research workflows that are not suited for a one-size-fits-all security approach. In the proposed model, network users and providers establish trust relationships that can be used to temporarily bypass the policy compliance checks applied to general-purpose traffic -- typically by network appliances that perform Deep Packet Inspection, thereby creating network bottlenecks. We describe the components of a prototype exception system as well as experiments showing that through short-lived exceptions researchers can realize significant improvements for their special-purpose traffic