A novel approach for dynamic capacity sharing in multi-tenant scenarios

Network slicing is included as a key feature of the 5G architecture in order to simultaneously support diverse service types with heterogeneous requirements. The realization of network slicing in the Radio Access Network (RAN) needs mechanisms that allow the distribution of the available capacity in the system in an efficient manner while satisfying the requirements of the different services. In this paper, a capacity sharing function is proposed, which is approached as a multi agent reinforcement learning based on the Deep Reinforcement Learning (DRL) algorithm Deep Q-Network (DQN). The proposed algorithm provides the capacity to be assigned to each RAN slice. Performance assessment reveals the promising behavior of the proposed solution.This work has been supported by the Spanish Research Council and FEDER funds under SONAR 5G grant (ref. TEC2017-82651-R), by the European Commission’s Horizon 2020 research and innovation program under grant agreement #871428, 5G-CLARITY project, and by the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia under grant 2019FI_B1 00102.Peer ReviewedPostprint (author's final draft

Evaluation of a multi-cell and multi-tenant capacity sharing solution under heterogeneous traffic distributions

One of the key features of the 5G architecture is network slicing, which allows the simultaneous support of diverse service types with heterogeneous requirements over a common network infrastructure. In order to support this feature in the Radio Access Network (RAN), it is required to have capacity sharing mechanisms that distribute the available capacity in each cell among the existing RAN slices while satisfying their requirements and efficiently using the available resources. Deep Reinforcement Learning (DRL) techniques are good candidates to deal with the complexity of capacity sharing in multi-cell scenarios where the traffic in the different cells can be heterogeneously distributed in the time and space domains. In this paper, a multi-agent reinforcement learning-based solution for capacity sharing in multi-cell scenarios is discussed and assessed under heterogeneous traffic conditions. Results show the capability of the solution to satisfy the requirements of the RAN slices while using the resources in the different cells efficiently.This work has been supported by the Spanish Research Council and FEDER funds under SONAR 5G grant (ref.TEC2017-82651-R), by the European Commission’s Horizon 2020 5G-CLARITY project under grant agreement 871428 and by the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia under grant 2020FI_B2 00075.Peer ReviewedPostprint (author's final draft

Contribution to the modelling and evaluation of radio network slicing solutions in 5G

Network slicing is a key feature of 5G architecture that allows the partitioning of the network into multiple logical networks, known as network slices, where each of them is customised according to the specific needs of a service or application. Thus, network slicing allows the materialisation of multi-tenant networks, in which a common network infrastructure is shared among multiple communication providers, acting as tenants and each of them using a different network slice. The support of multi-tenancy through slicing in the Radio Access Network (RAN), known as RAN slicing, is particularly challenging because it involves the configuration and operation of multiple and diverse RAN behaviours over the common pool of radio resources available at each of the RAN nodes. Moreover, this configuration needs to be performed in such a way that the specific requirements of each tenant are satisfied and, at the same time, the available radio resources are efficiently used. Therefore, new functionalities that allow the deployment of RAN slices are needed to be introduced at different levels, ranging from Radio Resource Management (RRM) functionalities that incorporate RAN slicing parameters to functionalities that support the lifecycle management of RAN slices. This thesis has addressed this need by proposing, developing and assessing diverse solutions for the support RAN slicing, which has allowed evaluating the capacities, requirements and limitations of network slicing in the RAN from diverse perspectives. Specifically, this thesis is firstly focused on the analytical assessment of RRM functionalities that support multi-tenant and multi-services scenarios, where services are defined according to their 5G QoS requirements. This assessment is conducted through the Markov modelling of admission control policies and the statistical modelling of the resourc allocation, both supporting multiple tenants and multiple services. Secondly, the thesis addresses the problem of slice admission control by proposing a methodology for the estimation of the radio resources required by a RAN slice based on data analytics. This methodology supports the decision on the admission or rejection of new RAN slice creation requests. Thirdly, the thesis explores the potential of artificial intelligence, and specifically, of Deep Reinforcement Learning (DRL) to deal with the capacity sharing problem in RAN slicing scenarios. To this end, a DRL-based capacity sharing solution that distributes the available capacity of a multi-cell scenario among multiple tenants is proposed and assessed. The solution consists in a Multi-Agent Reinforcement Learning (MARL) approach based on Deep Q-Network. Finally, this thesis discuses diverse implementation aspects of the DRL-based capacity sharing solution, including considerations on its compatibility with the standards, the impact of the training on the achieved performance, as well as the tools and technologies required for the deployment of the solution in the real network environment.El Network Slicing és una tecnologia clau de l’arquitectura del 5G que permet dividir la xarxa en múltiples xarxes lògiques, conegudes com a network slices, on cada una es configura d’acord a les necessitats d’un servei o aplicació específic. Així, el network slicing permet la materialització de les xarxes amb múltiples inquilins, on una infraestructura de xarxa comuna es comparteix entre diferents proveïdors de comunicacions, que actuen com a inquilins i utilitzen network slices diferents. El suport de múltiples inquilins mitjançant l’ús del network slicing a la xarxa d’accés ràdio (RAN), que es coneix com a RAN slicing, és un gran repte tecnològic, ja que comporta la configuració i operació de múltiples i diversos comportaments sobre els recursos ràdio disponibles a cadascun dels nodes de la xarxa d’accés. A més a més, aquesta configuració s’ha de portar a terme de forma que els requisits específics de cada inquilí es satisfacin i, al mateix temps, els recursos ràdio disponibles s’utilitzin eficientment. Per tant, és necessari introduir noves funcionalitats a diferents nivells que permetin el desplegament de les RAN slices, des de funcionalitats relacionades amb la gestió dels recursos ràdio (RRM) que incorporin paràmetres per al RAN slicing a funcionalitats que proporcionin suport a la gestió del cicle de vida de les RAN slices. Aquesta tesi ha adreçat aquesta necessitat proposant, desenvolupant i avaluant diverses solucions pel suport del RAN slicing, que han permès analitzar les capacitats, requisits i limitacions del RAN slicing des de diferents perspectives. Específicament, aquesta tesi es centra, en primer lloc, en realitzar una anàlisi de les funcionalitats de RRM que suporten escenaris amb múltiples inquilins i múltiples serveis, on els serveis es defineixen d’acord amb els seus requisits de 5G QoS. Aquesta anàlisi es porta a terme mitjançant la caracterització de polítiques de control d’admissió amb un model de Markov i el modelat estadístic de l’assignació de recursos, ambdós suportant múltiples inquilins i múltiples serveis. En segon lloc, la tesi aborda el problema del control d’admissió de network slices proposant una metodologia per l¿estimació dels recursos requerits per una RAN slice, que es basa en la anàlisi de dades. Aquesta metodologia dona suport a la decisió sobre l’admissió o rebuig de noves sol·licituds de creació de RAN slices. En tercer lloc, la tesi explora el potencial de la intel·ligència artificial, concretament, de les tècniques de Deep Reinforcement Learning (DRL) per a tractar el problema de la compartició de capacitat en escenaris amb RAN slicing. Amb aquest objectiu, es proposa i s’avalua una solució de compartició de capacitat basada en DRL que distribueix la capacitat disponible en un escenari multicel·lular entre múltiples inquilins. Aquesta solució es planteja com una solución de Multi-Agent Reinforcement Learning (MARL) basat en Deep Q-Network. Finalment, aquesta tesi tracta diversos aspectes relacionats amb la implementació de la solució de compartició de capacitat basada en DRL, incloent-hi consideracions sobre la compatibilitat de la solució amb els estàndards, l’impacte de l’entrenament de la solució al seu comportament i rendiment, així com les eines i tecnologies necessàries per al desplegament de la solució en un entorn de xarxa real.El Network Slicing es una tecnología clave de la arquitectura del 5G que permite dividir la red en múltiples redes lógicas, conocidas como network slices, que se configuran de acuerdo a las necesidades de servicios y aplicaciones específicas. Así, el network slicing permite la materialización de las redes con múltiples inquilinos, donde una infraestructura de red común se comparte entre diferentes proveedores de comunicaciones, que actúan como inquilinos y que usan network slices diferentes. El soporte de múltiples inquilinos mediante el uso del network slicing en la red de acceso radio (RAN), que se conoce como RAN slicing, es un gran reto tecnológico, ya que comporta la configuración y operación de múltiples y diversos comportamientos sobre los recursos radio disponibles en cada uno de los nodos de la red de acceso. Además, esta configuración debe realizarse de tal manera que los requisitos específicos de cada inquilino se satisfagan y, al mismo tiempo, los recursos radio disponibles se utilicen eficazmente. Por lo tanto, es necesario introducir nuevas funcionalidades a diferentes niveles que permitan el despliegue de las RAN slices, desde funcionalidades relacionadas con la gestión de recursos radio (RRM) que incorporen parámetros para el RAN slicing a funcionalidades que proporcionen soporte a la gestión del ciclo de vida de las RAN slices. Esta tesis ha abordado esta necesidad proponiendo, desarrollando y evaluando diversas soluciones para el soporte del RAN slicing, lo que ha permitido analizar las capacidades, requisitos y limitaciones del RAN slicing desde diversas perspectivas. Específicamente, esta tesis se centra, en primer lugar, en realizar un análisis de funcionalidades de RRM que soportan escenarios con múltiples inquilinos y múltiples servicios, donde los servicios se definen según sus requisitos de 5G QoS. Este análisis se lleva a cabo mediante la caracterización de políticas de control de admisión mediante un modelo de Markov y el modelado a nivel estadístico de la asignación de recursos, ambos soportando múltiples inquilinos y múltiples servicios. En segundo lugar, la tesis aborda el problema del control de admisión de network slices proponiendo una metodología para la estimación de los recursos radio requeridos por una RAN slice que se basa en análisis de datos. Esta metodología da soporte a la decisión sobre la admisión o el rechazo de nuevas solicitudes de creación de RAN slice. En tercer lugar, la tesis explora el potencial de la inteligencia artificial, y en concreto, de las técnicas de Deep Reinforcement Learning (DRL) para tratar el problema de compartición de capacidad en escenarios de RAN slicing. Para ello, se propone y evalúa una solución de compartición de capacidad basada en DRL que distribuye la capacidad disponible de un escenario multicelular entre múltiples inquilinos. Esta solución se plantea como una solución de Multi-Agent Reinforcement Learning (MARL) basado en Deep Q-Network. Finalmente, en esta tesis se tratan diversos aspectos relacionados con la implementación de la solución de reparto de capacidad basada en DRL, incluyendo consideraciones sobre su compatibilidad con los estándares, el impacto del entrenamiento en el comportamiento y rendimiento conseguido, así como las herramientas y tecnologías necesarias para su despliegue en un entorno de red real.Postprint (published version

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications

Applications

Volume 3 describes how resource-aware machine learning methods and techniques are used to successfully solve real-world problems. The book provides numerous specific application examples: in health and medicine for risk modelling, diagnosis, and treatment selection for diseases in electronics, steel production and milling for quality control during manufacturing processes in traffic, logistics for smart cities and for mobile communications

A patient agent controlled customized blockchain based framework for internet of things

Although Blockchain implementations have emerged as revolutionary technologies for various industrial applications including cryptocurrencies, they have not been widely deployed to store data streaming from sensors to remote servers in architectures known as Internet of Things. New Blockchain for the Internet of Things models promise secure solutions for eHealth, smart cities, and other applications. These models pave the way for continuous monitoring of patient’s physiological signs with wearable sensors to augment traditional medical practice without recourse to storing data with a trusted authority. However, existing Blockchain algorithms cannot accommodate the huge volumes, security, and privacy requirements of health data. In this thesis, our first contribution is an End-to-End secure eHealth architecture that introduces an intelligent Patient Centric Agent. The Patient Centric Agent executing on dedicated hardware manages the storage and access of streams of sensors generated health data, into a customized Blockchain and other less secure repositories. As IoT devices cannot host Blockchain technology due to their limited memory, power, and computational resources, the Patient Centric Agent coordinates and communicates with a private customized Blockchain on behalf of the wearable devices. While the adoption of a Patient Centric Agent offers solutions for addressing continuous monitoring of patients’ health, dealing with storage, data privacy and network security issues, the architecture is vulnerable to Denial of Services(DoS) and single point of failure attacks. To address this issue, we advance a second contribution; a decentralised eHealth system in which the Patient Centric Agent is replicated at three levels: Sensing Layer, NEAR Processing Layer and FAR Processing Layer. The functionalities of the Patient Centric Agent are customized to manage the tasks of the three levels. Simulations confirm protection of the architecture against DoS attacks. Few patients require all their health data to be stored in Blockchain repositories but instead need to select an appropriate storage medium for each chunk of data by matching their personal needs and preferences with features of candidate storage mediums. Motivated by this context, we advance third contribution; a recommendation model for health data storage that can accommodate patient preferences and make storage decisions rapidly, in real-time, even with streamed data. The mapping between health data features and characteristics of each repository is learned using machine learning. The Blockchain’s capacity to make transactions and store records without central oversight enables its application for IoT networks outside health such as underwater IoT networks where the unattended nature of the nodes threatens their security and privacy. However, underwater IoT differs from ground IoT as acoustics signals are the communication media leading to high propagation delays, high error rates exacerbated by turbulent water currents. Our fourth contribution is a customized Blockchain leveraged framework with the model of Patient-Centric Agent renamed as Smart Agent for securely monitoring underwater IoT. Finally, the smart Agent has been investigated in developing an IoT smart home or cities monitoring framework. The key algorithms underpinning to each contribution have been implemented and analysed using simulators.Doctor of Philosoph

A Polyhedral Study of Mixed 0-1 Set

We consider a variant of the well-known single node fixed charge network flow set with constant capacities. This set arises from the relaxation of more general mixed integer sets such as lot-sizing problems with multiple suppliers. We provide a complete polyhedral characterization of the convex hull of the given set

Fuelling the zero-emissions road freight of the future: routing of mobile fuellers

The future of zero-emissions road freight is closely tied to the sufficient availability of new and clean fuel options such as electricity and Hydrogen. In goods distribution using Electric Commercial Vehicles (ECVs) and Hydrogen Fuel Cell Vehicles (HFCVs) a major challenge in the transition period would pertain to their limited autonomy and scarce and unevenly distributed refuelling stations. One viable solution to facilitate and speed up the adoption of ECVs/HFCVs by logistics, however, is to get the fuel to the point where it is needed (instead of diverting the route of delivery vehicles to refuelling stations) using "Mobile Fuellers (MFs)". These are mobile battery swapping/recharging vans or mobile Hydrogen fuellers that can travel to a running ECV/HFCV to provide the fuel they require to complete their delivery routes at a rendezvous time and space. In this presentation, new vehicle routing models will be presented for a third party company that provides MF services. In the proposed problem variant, the MF provider company receives routing plans of multiple customer companies and has to design routes for a fleet of capacitated MFs that have to synchronise their routes with the running vehicles to deliver the required amount of fuel on-the-fly. This presentation will discuss and compare several mathematical models based on different business models and collaborative logistics scenarios