Orchestration of distributed ingestion and processing of IoT data for fog platforms

In recent years there has been an extraordinary growth of the Internet of Things (IoT) and its protocols. The increasing diffusion of electronic devices with identification, computing and communication capabilities is laying ground for the emergence of a highly distributed service and networking environment. The above mentioned situation implies that there is an increasing demand for advanced IoT data management and processing platforms. Such platforms require support for multiple protocols at the edge for extended connectivity with the objects, but also need to exhibit uniform internal data organization and advanced data processing capabilities to fulfill the demands of the application and services that consume IoT data. One of the initial approaches to address this demand is the integration between IoT and the Cloud computing paradigm. There are many benefits of integrating IoT with Cloud computing. The IoT generates massive amounts of data, and Cloud computing provides a pathway for that data to travel to its destination. But today’s Cloud computing models do not quite fit for the volume, variety, and velocity of data that the IoT generates. Among the new technologies emerging around the Internet of Things to provide a new whole scenario, the Fog Computing paradigm has become the most relevant. Fog computing was introduced a few years ago in response to challenges posed by many IoT applications, including requirements such as very low latency, real-time operation, large geo-distribution, and mobility. Also this low latency, geo-distributed and mobility environments are covered by the network architecture MEC (Mobile Edge Computing) that provides an IT service environment and Cloud-computing capabilities at the edge of the mobile network, within the Radio Access Network (RAN) and in close proximity to mobile subscribers. Fog computing addresses use cases with requirements far beyond Cloud-only solution capabilities. The interplay between Cloud and Fog computing is crucial for the evolution of the so-called IoT, but the reach and specification of such interplay is an open problem. This thesis aims to find the right techniques and design decisions to build a scalable distributed system for the IoT under the Fog Computing paradigm to ingest and process data. The final goal is to explore the trade-offs and challenges in the design of a solution from Edge to Cloud to address opportunities that current and future technologies will bring in an integrated way. This thesis describes an architectural approach that addresses some of the technical challenges behind the convergence between IoT, Cloud and Fog with special focus on bridging the gap between Cloud and Fog. To that end, new models and techniques are introduced in order to explore solutions for IoT environments. This thesis contributes to the architectural proposals for IoT ingestion and data processing by 1) proposing the characterization of a platform for hosting IoT workloads in the Cloud providing multi-tenant data stream processing capabilities, the interfaces over an advanced data-centric technology, including the building of a state-of-the-art infrastructure to evaluate the performance and to validate the proposed solution. 2) studying an architectural approach following the Fog paradigm that addresses some of the technical challenges found in the first contribution. The idea is to study an extension of the model that addresses some of the central challenges behind the converge of Fog and IoT. 3) Design a distributed and scalable platform to perform IoT operations in a moving data environment. The idea after study data processing in Cloud, and after study the convenience of the Fog paradigm to solve the IoT close to the Edge challenges, is to define the protocols, the interfaces and the data management to solve the ingestion and processing of data in a distributed and orchestrated manner for the Fog Computing paradigm for IoT in a moving data environment.En els últims anys hi ha hagut un gran creixement del Internet of Things (IoT) i els seus protocols. La creixent difusió de dispositius electrònics amb capacitats d'identificació, computació i comunicació esta establint les bases de l’aparició de serveis altament distribuïts i del seu entorn de xarxa. L’esmentada situació implica que hi ha una creixent demanda de plataformes de processament i gestió avançada de dades per IoT. Aquestes plataformes requereixen suport per a múltiples protocols al Edge per connectivitat amb el objectes, però també necessiten d’una organització de dades interna i capacitats avançades de processament de dades per satisfer les demandes de les aplicacions i els serveis que consumeixen dades IoT. Una de les aproximacions inicials per abordar aquesta demanda és la integració entre IoT i el paradigma del Cloud computing. Hi ha molts avantatges d'integrar IoT amb el Cloud. IoT genera quantitats massives de dades i el Cloud proporciona una via perquè aquestes dades viatgin a la seva destinació. Però els models actuals del Cloud no s'ajusten del tot al volum, varietat i velocitat de les dades que genera l'IoT. Entre les noves tecnologies que sorgeixen al voltant del IoT per proporcionar un escenari nou, el paradigma del Fog Computing s'ha convertit en la més rellevant. Fog Computing es va introduir fa uns anys com a resposta als desafiaments que plantegen moltes aplicacions IoT, incloent requisits com baixa latència, operacions en temps real, distribució geogràfica extensa i mobilitat. També aquest entorn està cobert per l'arquitectura de xarxa MEC (Mobile Edge Computing) que proporciona serveis de TI i capacitats Cloud al edge per la xarxa mòbil dins la Radio Access Network (RAN) i a prop dels subscriptors mòbils. El Fog aborda casos d?us amb requisits que van més enllà de les capacitats de solucions només Cloud. La interacció entre Cloud i Fog és crucial per a l'evolució de l'anomenat IoT, però l'abast i especificació d'aquesta interacció és un problema obert. Aquesta tesi té com objectiu trobar les decisions de disseny i les tècniques adequades per construir un sistema distribuït escalable per IoT sota el paradigma del Fog Computing per a ingerir i processar dades. L'objectiu final és explorar els avantatges/desavantatges i els desafiaments en el disseny d'una solució des del Edge al Cloud per abordar les oportunitats que les tecnologies actuals i futures portaran d'una manera integrada. Aquesta tesi descriu un enfocament arquitectònic que aborda alguns dels reptes tècnics que hi ha darrere de la convergència entre IoT, Cloud i Fog amb especial atenció a reduir la bretxa entre el Cloud i el Fog. Amb aquesta finalitat, s'introdueixen nous models i tècniques per explorar solucions per entorns IoT. Aquesta tesi contribueix a les propostes arquitectòniques per a la ingesta i el processament de dades IoT mitjançant 1) proposant la caracterització d'una plataforma per a l'allotjament de workloads IoT en el Cloud que proporcioni capacitats de processament de flux de dades multi-tenant, les interfícies a través d'una tecnologia centrada en dades incloent la construcció d'una infraestructura avançada per avaluar el rendiment i validar la solució proposada. 2) estudiar un enfocament arquitectònic seguint el paradigma Fog que aborda alguns dels reptes tècnics que es troben en la primera contribució. La idea és estudiar una extensió del model que abordi alguns dels reptes centrals que hi ha darrere de la convergència de Fog i IoT. 3) Dissenyar una plataforma distribuïda i escalable per a realitzar operacions IoT en un entorn de dades en moviment. La idea després d'estudiar el processament de dades a Cloud, i després d'estudiar la conveniència del paradigma Fog per resoldre el IoT prop dels desafiaments Edge, és definir els protocols, les interfícies i la gestió de dades per resoldre la ingestió i processament de dades en un distribuït i orquestrat per al paradigma Fog Computing per a l'IoT en un entorn de dades en moviment

A Cognitive Routing framework for Self-Organised Knowledge Defined Networks

This study investigates the applicability of machine learning methods to the routing protocols for achieving rapid convergence in self-organized knowledge-defined networks. The research explores the constituents of the Self-Organized Networking (SON) paradigm for 5G and beyond, aiming to design a routing protocol that complies with the SON requirements. Further, it also exploits a contemporary discipline called Knowledge-Defined Networking (KDN) to extend the routing capability by calculating the “Most Reliable” path than the shortest one. The research identifies the potential key areas and possible techniques to meet the objectives by surveying the state-of-the-art of the relevant fields, such as QoS aware routing, Hybrid SDN architectures, intelligent routing models, and service migration techniques. The design phase focuses primarily on the mathematical modelling of the routing problem and approaches the solution by optimizing at the structural level. The work contributes Stochastic Temporal Edge Normalization (STEN) technique which fuses link and node utilization for cost calculation; MRoute, a hybrid routing algorithm for SDN that leverages STEN to provide constant-time convergence; Most Reliable Route First (MRRF) that uses a Recurrent Neural Network (RNN) to approximate route-reliability as the metric of MRRF. Additionally, the research outcomes include a cross-platform SDN Integration framework (SDN-SIM) and a secure migration technique for containerized services in a Multi-access Edge Computing environment using Distributed Ledger Technology. The research work now eyes the development of 6G standards and its compliance with Industry-5.0 for enhancing the abilities of the present outcomes in the light of Deep Reinforcement Learning and Quantum Computing

Architecture and algorithm for reliable 5G network design

This Ph.D. thesis investigates the resilient and cost-efficient design of both C-RAN and Xhaul architectures. Minimization of network resources as well as reuse of already deployed infrastructure, either based on fiber, wavelength, bandwidth or Processing Units (PU), is investigated and shown to be effective to reduce the overall cost. Moreover, the design of a survivable network against a single node (Baseband Unit hotel (BBU), Centralized/Distributed Unit (CU/DU) or link failure proposed. The novel function location algorithm, which adopts dynamic function chaining in relation to the evolution of the traffic estimation also proposed and showed remarkable improvement in terms of bandwidth saving and multiplexing gain with respect to conventional C-RAN. Finally, the adoption of Ethernet-based fronthaul and the introduction of hybrid switches is pursued to further decrease network cost by increasing optical resource usage

Contributing to the pathway towards 5G experimentation with an SDN-controlled network box

Καθώς η απαίτηση σε ευρυζωνικές υπηρεσίες κινητών επικοινωνιών αυξάνεται ραγδαία, τα υπάρχοντα δίκτυα κινητών επικοινωνιών πλησιάζουν τα όριά τους κάνοντας επιτακτική την ανάγκη εξέλιξής τους η οποία θα επέλθει με την τεχνολογική άφιξη της επόμενης γενιάς κινητών επικοινωνιών, ευρέως γνωστής ως 5G. Το 5G μεταφέρει όλες εκείνες τις δυνατότητες οι οποίες είναι απαραίτητες για να καλυφθούν οι συνεχώς αυξανόμενες ανάγκες σε ευρυζωνικές υπηρεσίες, να υποστηρίξουν το Internet of Things καθώς και να ενοποιήσουν ετερογενείς υπηρεσίες σε διαφορετικές βιομηχανίες. Η παρούσα διπλωματική εργασία στοχεύει να παρουσιάσει το “Network in a box”, ένα καινοτόμο εργαλείο που αναπτύξαμε στο εργαστήριο, το οποίο βασίζεται επάνω στους θεμέλιους λίθους του 5G, το SDN και το NFV. Με το SDN να είναι η νέα προσέγγιση στα δίκτυα κινητών επικοινωνιών, ο έλεγχος διαχωρίζεται από τα δεδομένα παρέχοντας τη δυνατότητα οποιεσδήποτε αποφάσεις ελέγχου, να λαμβάνονται κεντρικά, μετατρέποντας έτσι τις κλασικές δικτυακές συσκευές σε απλά προωθητικά στοιχεία του δικτύου. Η συγκεκριμένη διάταξη μιμείται ένα πραγματικό δίκτυο, το οποίο διαθέτει δυνατότητες αυτο-οργάνωσης και αυτο-βελτίωσης, προσομοιώνοντας τη λειτουργία του 5G δικτύου. Το συγκεκριμένο εργαλείο είναι επίσης ικανό να παράσχει KPI μετρικές του 5G δικτύου κάτω από πραγματικές συνθήκες ενόσω αληθινές δικτυακές συσκευές είναι συνδεδεμένες σε αυτό. Η δομή της παρούσας διπλωματικής εργασίας αναλύεται σε πέντε κεφάλαια. Το πρώτο κεφάλαιο παρουσιάζει τις προκλήσεις που σύντομα θα κληθούν να αντιμετωπίσουν τα δίκτυα κινητών επικοινωνιών και πώς αυτές μπορούν να καλυφθούν με την τεχνολογία του 5G. Το δεύτερο κεφάλαιο εισάγει την τάση στην αγορά των κινητών επικοινωνιών που διαφένεται πίσω από την επερχόμενη άφιξη του 5G, αποκαλύπτοντας το επιχειρηματικό πλαίσιο για επιχειρήσεις, καταναλωτές και συνεργασίες όπως επίσης και κάποιες περιπτώσεις χρήσης που αντικατοπτρίζουν την διαρκή εξέλιξη στις ευρυζωνικές υπηρεσίες κινητών επικοινωνιών. Το τρίτο κεφάλαιο εμπεριέχει μια μικρή επισκόπηση των τρέχοντων έργων πάνω στο 5G, τα οποία ξεκίνησαν υπό την αιγίδα της Ευρωπαϊκής Επιτροπής με τη συνεργασία προμηθευτών τεχνολογίας επικοινωνιών, παρόχων υπηρεσιών, μικρομεσαίων επιχειρήσεων και πανεπιστημίων. Γίνεται επίσης αναφορά στις βασικές τεχνολογίες του 5G και στις δραστηριότητες προτυποποίησής του. Προχωρώντας στο τέταρτο κεφάλαιο, περιγράφουμε σε βάθος την αρχιτεκτονική του 5G δικτύου, αναλύοντας τα SDN, NFV, MANO και εξετάζουμε πώς αυτά συνεισφέρουν στη βιωσιμότητα του δικτύου. Τέλος, στο πέμπτο κεφάλαιο εισάγουμε μια καινοτόμο ιδέα που αναπτύξαμε στο εργαστήριο δικτύων του πανεπιστημίου μας, ένα πλήρως αυτόνομο δικτυακό εργαλείο, το “Network in a box”. Παρουσιάζουμε σε βάθος πώς αυτός ο server μπορεί να εγκατασταθεί και να λειτουργήσει καθώς και τις δυνατότητές του κάτω από πραγματικές συνθήκες λειτουργίας του δικτύου, ενώ λαμβάνουν χώρα υποβάθμιση ποιότητας ή μη-διαθεσιμότητα στις δικτυακές ζεύξεις, παρέχοντας επίσης μετρικές από τη λειτουργία του δικτύου σε πραγματικό χρόνο.As the demand in mobile broadband is tremendously increased and the heterogeneity of the services to be covered is growing rapidly, current mobile networks are close to their limits imposing the need of an evolution which is going to be introduced by the next generation technology, the ITU IMT-2020, well known as 5G. 5G brings all those capabilities required to cover the increased mobile broadband needs, support the Internet of Things and bind heterogeneous services in different industries. This diploma thesis aims at presenting the “Network in a box”, an innovative tool we developed which is based on the key 5G principles, SDN and NFV. With Software Defined Networking (SDN) being the new approach in mobile networks, control and data plane are decoupled providing the ability to make any control related decisions centrally and transform legacy network devices to simple forwarding elements. This testbed is a portable emulated network device which is self-managed and self-optimised and can be connected between any real network devices, emulating how the 5G network will perform. This plug & play black-box testbed is also capable of providing KPI metrics of the 5G network under real circumstances when real network devices are connected to it. The structure of this diploma thesis is decomposed in five chapters. Chapter 1 presents the challenges mobile networks will shortly face due to the growing heterogeneous demands in communications towards the year 2020 and beyond and how these can be met with the upcoming 5G technology. Chapter 2 introduces the market trend behind the new era of 5G, revealing the business context for enterprises, consumers, verticals and partnerships as well as some use cases which reflect the continuous mobile broadband evolution. Chapter 3 includes a short overview of the ongoing 5G projects, initiated under the umbrella of the European Commission, with the collaboration of communications technology vendors, telecommunications operators, service providers, small and medium-sized enterprises (SMEs) and universities. There is also a reference in 5G key enabling technologies and standardisation activities as we move towards the next generation mobile networks technology. Moving forward, chapter 4 describes in detail the technological components of 5G network architecture such as SDN, NFV, MANO and examines how these 5G key enabling technologies contribute to the overall networks’ sustainability. Finally, in chapter 5 we introduce an innovative idea developed in our university’s communications network research laboratory, an autonomous emulated portable network testbed, the “Network in a box”. We present in-depth how this portable server is deployed, operates and demonstrate the way it can be connected to real network elements emulating a real 5G end-to-end customer network. Moreover, in this last chapter we present “Network in a box” capabilities under real network circumstances when link degradations or failures take place, providing also real-time network metrics

A Survey on Security and Privacy of 5G Technologies: Potential Solutions, Recent Advancements, and Future Directions

Security has become the primary concern in many telecommunications industries today as risks can have high consequences. Especially, as the core and enable technologies will be associated with 5G network, the confidential information will move at all layers in future wireless systems. Several incidents revealed that the hazard encountered by an infected wireless network, not only affects the security and privacy concerns, but also impedes the complex dynamics of the communications ecosystem. Consequently, the complexity and strength of security attacks have increased in the recent past making the detection or prevention of sabotage a global challenge. From the security and privacy perspectives, this paper presents a comprehensive detail on the core and enabling technologies, which are used to build the 5G security model; network softwarization security, PHY (Physical) layer security and 5G privacy concerns, among others. Additionally, the paper includes discussion on security monitoring and management of 5G networks. This paper also evaluates the related security measures and standards of core 5G technologies by resorting to different standardization bodies and provide a brief overview of 5G standardization security forces. Furthermore, the key projects of international significance, in line with the security concerns of 5G and beyond are also presented. Finally, a future directions and open challenges section has included to encourage future research.European CommissionNational Research Tomsk Polytechnic UniversityUpdate citation details during checkdate report - A

New Waves of IoT Technologies Research – Transcending Intelligence and Senses at the Edge to Create Multi Experience Environments

The next wave of Internet of Things (IoT) and Industrial Internet of Things (IIoT) brings new technological developments that incorporate radical advances in Artificial Intelligence (AI), edge computing processing, new sensing capabilities, more security protection and autonomous functions accelerating progress towards the ability for IoT systems to self-develop, self-maintain and self-optimise. The emergence of hyper autonomous IoT applications with enhanced sensing, distributed intelligence, edge processing and connectivity, combined with human augmentation, has the potential to power the transformation and optimisation of industrial sectors and to change the innovation landscape. This chapter is reviewing the most recent advances in the next wave of the IoT by looking not only at the technology enabling the IoT but also at the platforms and smart data aspects that will bring intelligence, sustainability, dependability, autonomy, and will support human-centric solutions.acceptedVersio