Computation Offloading in Beyond 5G Networks: A Distributed Learning Framework and Applications

Facing the trend of merging wireless communications and multi-access edge computing (MEC), this article studies computation offloading in the beyond fifth-generation networks. To address the technical challenges originating from the uncertainties and the sharing of limited resource in an MEC system, we formulate the computation offloading problem as a multi-agent Markov decision process, for which a distributed learning framework is proposed. We present a case study on resource orchestration in computation offloading to showcase the potentials of an online distributed reinforcement learning algorithm developed under the proposed framework. Experimental results demonstrate that our learning algorithm outperforms the benchmark resource orchestration algorithms. Furthermore, we outline the research directions worth in-depth investigation to minimize the time cost, which is one of the main practical issues that prevent the implementation of the proposed distributed learning framework

Reinforcement Learning in Self Organizing Cellular Networks

Self-organization is a key feature as cellular networks densify and become more heterogeneous, through the additional small cells such as pico and femtocells. Self- organizing networks (SONs) can perform self-configuration, self-optimization, and self-healing. These operations can cover basic tasks such as the configuration of a newly installed base station, resource management, and fault management in the network. In other words, SONs attempt to minimize human intervention where they use measurements from the network to minimize the cost of installation, configuration, and maintenance of the network. In fact, SONs aim to bring two main factors in play: intelligence and autonomous adaptability. One of the main requirements for achieving such goals is to learn from sensory data and signal measurements in networks. Therefore, machine learning techniques can play a major role in processing underutilized sensory data to enhance the performance of SONs. In the first part of this dissertation, we focus on reinforcement learning as a viable approach for learning from signal measurements. We develop a general framework in heterogeneous cellular networks agnostic to the learning approach. We design multiple reward functions and study different effects of the reward function, Markov state model, learning rate, and cooperation methods on the performance of reinforcement learning in cellular networks. Further, we look into the optimality of reinforcement learning solutions and provide insights into how to achieve optimal solutions. In the second part of the dissertation, we propose a novel architecture based on spatial indexing for system-evaluation of heterogeneous 5G cellular networks. We develop an open-source platform based on the proposed architecture that can be used to study large scale directional cellular networks. The proposed platform is used for generating training data sets of accurate signal-to-interference-plus-noise-ratio (SINR) values in millimeter-wave communications for machine learning purposes. Then, with taking advantage of the developed platform, we look into dense millimeter-wave networks as one of the key technologies in 5G cellular networks. We focus on topology management of millimeter-wave backhaul networks and study and provide multiple insights on the evaluation and selection of proper performance metrics in dense millimeter-wave networks. Finally, we finish this part by proposing a self-organizing solution to achieve k-connectivity via reinforcement learning in the topology management of wireless networks

Self organisation for 4G/5G networks

Nowadays, the rapid growth of mobile communications is changing the world towards a fully connected society. Current 4G networks account for almost half of total mobile traffic, and in the forthcoming years, the overall mobile data traffic is expected to dramatically increase. To manage this increase in data traffic, operators adopt network topologies such as Heterogeneous Networks. Thus, operators can de­ ploy hundreds of small cells for each macro cell, allowing them to reduce coverage hales and/or lack of capacity. The advent of this technology is expected to tremendously increase the number of nodes in this new ecosystem, so that traditional network management activities based on, e.g., classic manual and field trial design approaches are just not be viable anymore. As a consequence, the academic J literature has dedicated a significant amount of effort to Self-Organising Network (SON) algorithms. These solutions aim to bring intelligence and autonomous adaptability into cellular networks, thereby reducing capital and operation expenditures (CAPEX/OPEX). Another aspect to take into account is that, these type of networks generate a large amount of data during their normal operation in the form of control, management and data measurements. This data is expected to increase in SG due to different aspects, such as densification, heterogeneity in layers and technologies, additional control and management complexity in Network Functions Virtualisation (NFV) and Software Defined Network (SDN), and the advent of the Internet of Things (loT), among others. In this context, operators face the challenge of de ­ signing efficient technologies, while introducing new services, reaching challenges in terms networks, which are self-aware, self-adaptive, and intelligent. This dissertation provides a contribution to the design, analysis, and evaluation of SON solutions to improve network opera tor performance, expenses, and users' experience, by making the network more self-adaptive and intelligent. It also provides a contribution to the design of a self-aware network planning tool, which allows to predict the Quality of Service (QoS) offered to end-users, based on data al ­ ready available in the network . The main thesis contributions are divided into two parts. The first part presents a novel functional architecture based on an automatic and self-organised Reinforcement Learning (RL) based approach to model SON functionalities, in which the main task is the self-coordination of different actions taken by different SON functions to be automatically executed in a self-organised realistic Long Term Evolution (LTE) network. The proposed approach introduces a new paradigm to deal with the conflicts genera ted by the concurrent execution of multiple SON functions, revealing that the proposed approach is general enough to modelali the SON functions and their derived conflicts. The second part of the thesis is dedicated to the problem of QoS prediction. In particular, we aim at finding patterns of knowledge from physical layer data acquired from heterogeneous LTE networks. We propose an approach that not only is able to verify the QoS level experienced by the users, through physical layer measurements of the UEs, but it is a lso able to predict it based on measurements collected at different time, and from different regions of the heterogeneous network. We propose then to make predictions independently of the physical location, in order to exploit the experience gained in other sectors of the network, to properly dimension and deploy heterogeneous nodes. In this context, we use Machine Learning (ML) as a tool to allow the network to learn from experience, improving performances, and big data analytics to drive the network from reactive to predictive.Hoy en día, el rápido crecimiento de las comunicaciones móviles está cambiando el mundo hacia una sociedad completamente conectada. Las redes 4G actuales representan casi la mitad del tráfico móvil total, y en los próximos años se espera que el tráfico total de los dispositivos móviles aumente drásticamente. Para gestionar este incremento de tráfico de datos, los operadores adoptan tecnologías de redes como las redes heterogéneas. De esta manera, los operadores pueden desplegar centena res de pequeñas celdas por cada macro celda, permitiendo reducir zonas sin cobertura y/o falta de capacidad. Con la introducción de esta tecnología, se espera que incremente de manera sustancia l el número de nodos en el nuevo ecosistema, de manera que las actividades de gestión de las redes tradicionales, basadas en, por ejemplo, el diseño manual, sean inviables. Como consecuencia, la literatura académica ha dedicado un esfuerzo significativo al diseño de algoritmos de redes auto-organizadas (SON). Estas soluciones tienen como objetivo introducir inteligencia y capacidad autónoma a las redes móviles, reduciendo la capacidad y costes operativos. Otro aspecto a tener en cuenta es que este tipo de redes generan una gran cantidad de datos durante su funcionamiento habitual, en forma de medidas de control y gestión de datos. Se espera que estos datos incrementen con la tecnología SG, debido a diferentes aspectos como los son la densificación de redes heterogéneas, la complejidad adicional en el control y la gestión de la virtualización de las funciones de redes (NFV) y las redes definidas por software (SON), así como la llegada del internet de las cosas (loT), entre otros. En este contexto, los operadores se enfrentan al reto de diseñar tecnologías eficientes, mientras introducen nuevos servicios, consiguiendo objetivos en términos de satisfacción del cliente, en donde el objetivo global del operador es la construcción de redes auto-conscientes, auto-adaptables e inteligentes. Esta tesis ofrece una contribución al diseño y evaluación de soluciones SON para mejorar el rendimiento de las redes, los costes y la experiencia de los usuarios, consiguiendo que la red sea auto-adaptable e inteligente. Así mismo, proporciona una contribución al diseño de una herramienta de planificación de red auto-consciente, que permita predecir la calidad de servicio brindada a los usuarios finales, basada en la explotación de datos disponibles en la red.Avui en dia, el ràpid creixement de les comunicacions mòbils està canviant el món cap a una societat completament connectada. Les xarxes 4G actuals representen casi la m trànsit mòbil total, i en els propers anys s’espera que el trànsit total de dades mòbils augmenti dràsticament. Per gestionar aquest increment de trànsit de dades, els operadors adopten topologies de xarxa com ara les xarxes heterogènies (HetNets). D’aquesta manera, els operadors poden desplegar centenars de cel·les petites per a cada cella macro, permetent reduir forats en la cobertura i/o la manca de capacitat. Amb l’arribada d’aquesta tecnologia, s’espera que incrementi enormement el nombre de nodes en el nou ecosistema, de manera que les activitats de gestió de xarxa tradicionals, basades en, per exemple, el disseny manual i els assaigs de camp esdevenen simplement inviables. Com a conseqüència, la literatura acadèmica ha dedicat una quantitat significativa d’esforç als algorismes de xarxa auto organitzada (SON). Aquestes solucions tenen com a objectiu portar la intel·ligència i capacitat d’adaptació autònoma a les xarxes mòbils, reduint el capital i les despeses operatives (CAPES/OPEX). Un altre aspecte a tenir en compte és que aquest tipus de xarxes generen una gran quantitat de dades durant el seu funcionament habitual, en forma de mesuraments de control, gestió i dades. S’espera que aquestes dades incrementin amb la tecnologia 5G, degut a diferents aspectes com ara la densificació, l’heterogeneïtat en capes i tecnologies, la complexitat addicional en el control i la gestió de la virtualització de les funcions de xarxa (NFV) i xarxes definides per software (SDN), i l’adveniment de la internet de les coses (IoT), entre d’altres. En aquest context, els operadors s’enfronten al repte de dissenyar tecnologies eficients, mentre introdueixen nous serveis, aconseguint objectius en termes de satisfacció del client, i on l’objectiu global d’un operador és la construcció de xarxes que són autoconscients, auto-adaptables i intel·ligents. Aquesta tesis ofereix una contribució al disseny, l’anàlisi i l’avaluació de les solucions SON per millorar el rendiment de l’operador de xarxa, les xi despeses i l’experiència dels usuaris, fent que la xarxa sigui més auto-adaptable i intel·ligent. També proporciona una contribució al disseny d’una eina de planificació de xarxa autoconscient, el que permet predir la qualitat de servei (QoS) oferta als usuaris finals, basada en dades ja disponibles a la xarxa. Les contribucions principals d’aquesta tesis es divideixen en dues parts. La primera part presenta una nova arquitectura funcional basada en un aprenentatge per reforç (RL) automàtic i auto-organitzat, enfocat en modelar funcionalitats SON, on la tasca principal és l’auto-coordinació de les diferents accions dutes a terme perles diferents funcions SON a ser executades de forma automàtica en una xarxa Long Term Evolution (LTE) auto-organitzada. L’enfocament proposat introdueix un nou paradigma perfer front als conflictes generats per l’execució simultània de múltiples funcions SON, revelant que l’enfocament proposat és prou general per modelar totes les funcions SON i els seus conflictes derivats. La segona part de la tesis està dedicada al problema de la predicció de la qualitat de servei. En particular, el nostre objectiu és trobar patrons de coneixement a partir de dades de la capa física adquirides de xarxes LTE heterogènies. Proposem un enfocament que no només és capaç de verificar el nivell de QoS experimentat pels usuaris, a través de mesuraments de la capa física dels UEs, sinó que també és capaç de predir-ho basant-se en mesuraments adquirits en diferents instants, i de diferents regions de la xarxa heterogènia. Proposem per tant fer prediccions amb independència de la ubicació física, aprofitant l’experiència adquirida en altres sectors de la xarxa, per dimensionar i desplegar nodes heterogenis correctament. En aquest context, utilitzem l’aprenentatge automàtic (ML) com a eina per permetre que la xarxa aprengui de l’experiència, millorant el rendiment, i l’anàlisi de grans volums de dades per a conduir la xarxa de reactiva a predictiva. Durant l’elaboració d’aquesta tesis, s’han extret dues conclusions principals clau. En primer lloc, destaquem la importància de dissenyar algorismes SON eficients per fer front eficaçment a diversos reptes, com ara la ubicació més adequada de funcions SON i algorismes per resoldre adequadament el problema d’implementació distribuïda o centralitzada, o la solució de conflictes entre funcions SON executades a diferents nodes o xarxes. En segon lloc, en termes d’eines de planificació de xarxes, es poden trobar diferents eines cobrint una àmplia gamma de sistemes i aplicacions orientades a la indústria, així com per a fins d’investigació. En aquest context, les solucions investigades són sotmeses contínuament a canvis importants, on un del principals impulsors és presentar solucions més rentable

AI meets CRNs : a prospective review on the application of deep architectures in spectrum management

The spectrum low utilization and high demand conundrum created a bottleneck towards ful lling the requirements of next-generation networks. The cognitive radio (CR) technology was advocated as a de facto technology to alleviate the scarcity and under-utilization of spectrum resources by exploiting temporarily vacant spectrum holes of the licensed spectrum bands. As a result, the CR technology became the rst step towards the intelligentization of mobile and wireless networks, and in order to strengthen its intelligent operation, the cognitive engine needs to be enhanced through the exploitation of arti cial intelligence (AI) strategies. Since comprehensive literature reviews covering the integration and application of deep architectures in cognitive radio networks (CRNs) are still lacking, this article aims at lling the gap by presenting a detailed review that addresses the integration of deep architectures into the intricacies of spectrum management. This is a prospective review whose primary objective is to provide an in-depth exploration of the recent trends in AI strategies employed in mobile and wireless communication networks. The existing reviews in this area have not considered the relevance of incorporating the mathematical fundamentals of each AI strategy and how to tailor them to speci c mobile and wireless networking problems. Therefore, this reviewaddresses that problem by detailing howdeep architectures can be integrated into spectrum management problems. Beyond reviewing different ways in which deep architectures can be integrated into spectrum management, model selection strategies and how different deep architectures can be tailored into the CR space to achieve better performance in complex environments are then reported in the context of future research directions.The Sentech Chair in Broadband Wireless Multimedia Communications (BWMC) at the University of Pretoria.http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6287639am2022Electrical, Electronic and Computer Engineerin

Special Topics in Information Technology

This open access book presents thirteen outstanding doctoral dissertations in Information Technology from the Department of Electronics, Information and Bioengineering, Politecnico di Milano, Italy. Information Technology has always been highly interdisciplinary, as many aspects have to be considered in IT systems. The doctoral studies program in IT at Politecnico di Milano emphasizes this interdisciplinary nature, which is becoming more and more important in recent technological advances, in collaborative projects, and in the education of young researchers. Accordingly, the focus of advanced research is on pursuing a rigorous approach to specific research topics starting from a broad background in various areas of Information Technology, especially Computer Science and Engineering, Electronics, Systems and Control, and Telecommunications. Each year, more than 50 PhDs graduate from the program. This book gathers the outcomes of the thirteen best theses defended in 2020-21 and selected for the IT PhD Award. Each of the authors provides a chapter summarizing his/her findings, including an introduction, description of methods, main achievements and future work on the topic. Hence, the book provides a cutting-edge overview of the latest research trends in Information Technology at Politecnico di Milano, presented in an easy-to-read format that will also appeal to non-specialists

Definition and specification of connectivity and QoE/QoS management mechanisms – final report

This document summarizes the WP5 work throughout the project, describing its functional architecture and the solutions that implement the WP5 concepts on network control and orchestration. For this purpose, we defined 3 innovative controllers that embody the network slicing and multi tenancy: SDM-C, SDM-X and SDM-O. The functionalities of each block are detailed with the interfaces connecting them and validated through exemplary network processes, highlighting thus 5G NORMA innovations. All the proposed modules are designed to implement the functionality needed to provide the challenging KPIs required by future 5G networks while keeping the largest possible compatibility with the state of the art

Special Topics in Information Technology

This open access book presents thirteen outstanding doctoral dissertations in Information Technology from the Department of Electronics, Information and Bioengineering, Politecnico di Milano, Italy. Information Technology has always been highly interdisciplinary, as many aspects have to be considered in IT systems. The doctoral studies program in IT at Politecnico di Milano emphasizes this interdisciplinary nature, which is becoming more and more important in recent technological advances, in collaborative projects, and in the education of young researchers. Accordingly, the focus of advanced research is on pursuing a rigorous approach to specific research topics starting from a broad background in various areas of Information Technology, especially Computer Science and Engineering, Electronics, Systems and Control, and Telecommunications. Each year, more than 50 PhDs graduate from the program. This book gathers the outcomes of the thirteen best theses defended in 2020-21 and selected for the IT PhD Award. Each of the authors provides a chapter summarizing his/her findings, including an introduction, description of methods, main achievements and future work on the topic. Hence, the book provides a cutting-edge overview of the latest research trends in Information Technology at Politecnico di Milano, presented in an easy-to-read format that will also appeal to non-specialists

Machine Learning-based Orchestration Solutions for Future Slicing-Enabled Mobile Networks

The fifth generation mobile networks (5G) will incorporate novel technologies such as network programmability and virtualization enabled by Software-Defined Networking (SDN) and Network Function Virtualization (NFV) paradigms, which have recently attracted major interest from both academic and industrial stakeholders. Building on these concepts, Network Slicing raised as the main driver of a novel business model where mobile operators may open, i.e., “slice”, their infrastructure to new business players and offer independent, isolated and self-contained sets of network functions and physical/virtual resources tailored to specific services requirements. While Network Slicing has the potential to increase the revenue sources of service providers, it involves a number of technical challenges that must be carefully addressed. End-to-end (E2E) network slices encompass time and spectrum resources in the radio access network (RAN), transport resources on the fronthauling/backhauling links, and computing and storage resources at core and edge data centers. Additionally, the vertical service requirements’ heterogeneity (e.g., high throughput, low latency, high reliability) exacerbates the need for novel orchestration solutions able to manage end-to-end network slice resources across different domains, while satisfying stringent service level agreements and specific traffic requirements. An end-to-end network slicing orchestration solution shall i) admit network slice requests such that the overall system revenues are maximized, ii) provide the required resources across different network domains to fulfill the Service Level Agreements (SLAs) iii) dynamically adapt the resource allocation based on the real-time traffic load, endusers’ mobility and instantaneous wireless channel statistics. Certainly, a mobile network represents a fast-changing scenario characterized by complex spatio-temporal relationship connecting end-users’ traffic demand with social activities and economy. Legacy models that aim at providing dynamic resource allocation based on traditional traffic demand forecasting techniques fail to capture these important aspects. To close this gap, machine learning-aided solutions are quickly arising as promising technologies to sustain, in a scalable manner, the set of operations required by the network slicing context. How to implement such resource allocation schemes among slices, while trying to make the most efficient use of the networking resources composing the mobile infrastructure, are key problems underlying the network slicing paradigm, which will be addressed in this thesis

Robotix-Academy Conference for Industrial Robotics (RACIR) 2019

Robotix-Academy Conference for Industrial Robotics (RACIR) is held in University of Liège, Belgium, during June 05, 2019. The topics concerned by RACIR are: robot design, robot kinematics/dynamics/control, system integration, sensor/ actuator networks, distributed and cloud robotics, bio-inspired systems, service robots, robotics in automation, biomedical applications, autonomous vehicles (land, sea and air), robot perception, manipulation with multi-finger hands, micro/nano systems, sensor information, robot vision, multimodal interface and human-robot interaction.