Traffic control for energy harvesting virtual small cells via reinforcement learning

Due to the rapid growth of mobile data traffic, future mobile networks are expected to support at least 1000 times more capacity than 4G systems. This trend leads to an increasing energy demand from mobile networks which raises both economic and environmental concerns. Energy costs are becoming an important part of OPEX by Mobile Network Operators (MNOs). As a result, the shift towards energy-oriented design and operation of 5G and beyond systems has been emphasized by academia, industries as well as standard bodies. In particular, Radio Access Network (RAN) is the major energy consuming part of cellular networks. To increase the RAN efficiency, Cloud Radio Access Network (CRAN) has been proposed to enable centralized cloud processing of baseband functions while Base Stations (BSs) are reduced to simple Radio Remote Heads (RRHs). The connection between the RRHs and central cloud is provided by high capacity and very low latency fronthaul. Flexible functional splits between local BS sites and a central cloud are then proposed to relax the CRAN fronthaul requirements via partial processing of baseband functions at the local BS sites. Moreover, Network Function Virtualization (NFV) and Software Defined Networking (SDN) enable flexibility in placement and control of network functions. Relying on SDN/NFV with flexible functional splits, network functions of small BSs can be virtualized and placed at different sites of the network. These small BSs are known as virtual Small Cells (vSCs). More recently, Multi-access Edge Computing (MEC) has been introduced where BSs can leverage cloud computing capabilities and offer computational resources on demand basis. On the other hand, Energy Harvesting (EH) is a promising technology ensuring both cost effectiveness and carbon footprint reduction. However, EH comes with challenges mainly due to intermittent and unreliable energy sources. In EH Base Stations (EHBSs), it is important to intelligently manage the harvested energy as well as to ensure energy storage provision. Consequently, MEC enabled EHBSs can open a new frontier in energy-aware processing and sharing of processing units according to flexible functional split options. The goal of this PhD thesis is to propose energy-aware control algorithms in EH powered vSCs for efficient utilization of harvested energy and lowering the grid energy consumption of RAN, which is the most power consuming part of the network. We leverage on virtualization and MEC technologies for dynamic provision of computational resources according to functional split options employed by the vSCs. After describing the state-of-the-art, the first part of the thesis focuses on offline optimization for efficient harvested energy utilization via dynamic functional split control in vSCs powered by EH. For this purpose, dynamic programming is applied to determine the performance bound and comparison is drawn against static configurations. The second part of the thesis focuses on online control methods where reinforcement learning based controllers are designed and evaluated. In particular, more focus is given towards the design of multi-agent reinforcement learning to overcome the limitations of centralized approaches due to complexity and scalability. Both tabular and deep reinforcement learning algorithms are tailored in a distributed architecture with emphasis on enabling coordination among the agents. Policy comparison among the online controllers and against the offline bound as well as energy and cost saving benefits are also analyzed.Debido al rápido crecimiento del tráfico de datos móviles, se espera que las redes móviles futuras admitan al menos 1000 veces más capacidad que los sistemas 4G. Esta tendencia lleva a una creciente demanda de energía de las redes móviles, lo que plantea preocupaciones económicas y ambientales. Los costos de energía se están convirtiendo en una parte importante de OPEX por parte de los operadores de redes móviles (MNO). Como resultado, la academia, las industrias y los organismos estándar han enfatizado el cambio hacia el diseño orientado a la energía y la operación de sistemas 5G y más allá de los sistemas. En particular, la red de acceso por radio (RAN) es la principal parte de las redes celulares que consume energía. Para aumentar la eficiencia de la RAN, se ha propuesto Cloud Radio Access Network (CRAN) para permitir el procesamiento centralizado en la nube de las funciones de banda base, mientras que las estaciones base (BS) se reducen a simples cabezales remotos de radio (RRH). La conexión entre los RRHs y la nube central es proporcionada por una capacidad frontal de muy alta latencia y muy baja latencia. Luego se proponen divisiones funcionales flexibles entre los sitios de BS locales y una nube central para relajar los requisitos de red de enlace CRAN a través del procesamiento parcial de las funciones de banda base en los sitios de BS locales. Además, la virtualización de funciones de red (NFV) y las redes definidas por software (SDN) permiten flexibilidad en la colocación y el control de las funciones de red. Confiando en SDN / NFV con divisiones funcionales flexibles, las funciones de red de pequeñas BS pueden virtualizarse y ubicarse en diferentes sitios de la red. Estas pequeñas BS se conocen como pequeñas celdas virtuales (vSC). Más recientemente, se introdujo la computación perimetral de acceso múltiple (MEC) donde los BS pueden aprovechar las capacidades de computación en la nube y ofrecer recursos computacionales según la demanda. Por otro lado, Energy Harvesting (EH) es una tecnología prometedora que garantiza tanto la rentabilidad como la reducción de la huella de carbono. Sin embargo, EH presenta desafíos principalmente debido a fuentes de energía intermitentes y poco confiables. En las estaciones base EH (EHBS), es importante administrar de manera inteligente la energía cosechada, así como garantizar el suministro de almacenamiento de energía. En consecuencia, los EHBS habilitados para MEC pueden abrir una nueva frontera en el procesamiento con conciencia energética y el intercambio de unidades de procesamiento de acuerdo con las opciones de división funcional flexible. El objetivo de esta tesis doctoral es proponer algoritmos de control conscientes de la energía en vSC alimentados por EH para la utilización eficiente de la energía cosechada y reducir el consumo de energía de la red de RAN, que es la parte más consumidora de la red. Aprovechamos las tecnologías de virtualización y MEC para la provisión dinámica de recursos computacionales de acuerdo con las opciones de división funcional empleadas por los vSC. La primera parte de la tesis se centra en la optimización fuera de línea para la utilización eficiente de la energía cosechada a través del control dinámico de división funcional en vSC con tecnología EH. Para este propósito, la programación dinámica se aplica para determinar el rendimiento limitado y la comparación se realiza con configuraciones estáticas. La segunda parte de la tesis se centra en los métodos de control en línea donde se diseñan y evalúan los controladores basados en el aprendizaje por refuerzo. En particular, se presta más atención al diseño de aprendizaje de refuerzo de múltiples agentes para superar las limitaciones de los enfoques centralizados debido a la complejidad y la escalabilidad. También se analiza la comparación de políticas entre los controladores en línea y contra los límites fuera de línea,Postprint (published version

Enabling Technology and Proof-of-Concept Evaluation for RAN Architectural Migration toward 5G and Beyond Mobile Systems

In this paper, we address two major issues regarding architectural migration of radio access network (RAN). Firstly, an overview and explicit interpretation of how different enabling technologies over generations are brought up and coordinated for migration from a distributed, to a centralized, and then to a virtualized RAN for 5G and beyond cellular; and secondly, the proof-of-concept (PoC) evaluation to understand the feasibility of these enabling technologies, are addressed. In doing so, we first give an overview of major enabling technologies and discuss their impact on RAN migration. We then evaluate the PoC of major enabling technologies proposed mainly for 5G CRAN, namely functional split options, TDM-PON systems, and virtualization techniques using a mobile CORD based prototype in LTE systems with ideal fronthauls. PoC experimental results with split options 2 and 5 are presented and compared using TCP and UDP traffic. Experimentally, it is shown that the throughput improvement is significant for TCP as compared to UDP with virtualized BBUs, which are about 30%-40% and 40%-45% higher in mean throughputs respectively in downlink and uplink with split 5 than that with split 2. Finally, we point out the major experimental limitations of PoC and future research directions

Sparse Signal Processing Concepts for Efficient 5G System Design

As it becomes increasingly apparent that 4G will not be able to meet the emerging demands of future mobile communication systems, the question what could make up a 5G system, what are the crucial challenges and what are the key drivers is part of intensive, ongoing discussions. Partly due to the advent of compressive sensing, methods that can optimally exploit sparsity in signals have received tremendous attention in recent years. In this paper we will describe a variety of scenarios in which signal sparsity arises naturally in 5G wireless systems. Signal sparsity and the associated rich collection of tools and algorithms will thus be a viable source for innovation in 5G wireless system design. We will discribe applications of this sparse signal processing paradigm in MIMO random access, cloud radio access networks, compressive channel-source network coding, and embedded security. We will also emphasize important open problem that may arise in 5G system design, for which sparsity will potentially play a key role in their solution.Comment: 18 pages, 5 figures, accepted for publication in IEEE Acces

Efficient sharing mechanisms for virtualized multi-tenant heterogeneous networks

The explosion in data traffic, the physical resource constraints, and the insufficient financial incentives for deploying 5G networks, stress the need for a paradigm shift in network upgrades. Typically, operators are also the service providers, which charge the end users with low and flat tariffs, independently of the service enjoyed. A fine-scale management of the network resources is needed, both for optimizing costs and resource utilization, as well as for enabling new synergies among network owners and third-parties. In particular, operators could open their networks to third parties by means of fine-scale sharing agreements over customized networks for enhanced service provision, in exchange for an adequate return of investment for upgrading their infrastructures. The main objective of this thesis is to study the potential of fine-scale resource management and sharing mechanisms for enhancing service provision and for contributing to a sustainable road to 5G. More precisely, the state-of-the-art architectures and technologies for network programmability and scalability are studied, together with a novel paradigm for supporting service diversity and fine-scale sharing. We review the limits of conventional networks, we extend existing standardization efforts and define an enhanced architecture for enabling 5G networks' features (e.g., network-wide centralization and programmability). The potential of the proposed architecture is assessed in terms of flexible sharing and enhanced service provision, while the advantages of alternative business models are studied in terms of additional profits to the operators. We first study the data rate improvement achievable by means of spectrum and infrastructure sharing among operators and evaluate the profit increase justified by a better service provided. We present a scheme based on coalitional game theory for assessing the capability of accommodating more service requests when a cooperative approach is adopted, and for studying the conditions for beneficial sharing among coalitions of operators. Results show that: i) collaboration can be beneficial also in case of unbalanced cost redistribution within coalitions; ii) coalitions of equal-sized operators provide better profit opportunities and require lower tariffs. The second kind of sharing interaction that we consider is the one between operators and third-party service providers, in the form of fine-scale provision of customized portions of the network resources. We define a policy-based admission control mechanism, whose performance is compared with reference strategies. The proposed mechanism is based on auction theory and computes the optimal admission policy at a reduced complexity for different traffic loads and allocation frequencies. Because next-generation services include delay-critical services, we compare the admission control performances of conventional approaches with the proposed one, which proves to offer near real-time service provision and reduced complexity. Besides, it guarantees high revenues and low expenditures in exchange for negligible losses in terms of fairness towards service providers. To conclude, we study the case where adaptable timescales are adopted for the policy-based admission control, in order to promptly guarantee service requirements over traffic fluctuations. In order to reduce complexity, we consider the offline pre­computation of admission strategies with respect to reference network conditions, then we study the extension to unexplored conditions by means of computationally efficient methodologies. Performance is compared for different admission strategies by means of a proof of concept on real network traces. Results show that the proposed strategy provides a tradeoff in complexity and performance with respect to reference strategies, while reducing resource utilization and requirements on network awareness.La explosion del trafico de datos, los recursos limitados y la falta de incentivos para el desarrollo de 5G evidencian la necesidad de un cambio de paradigma en la gestion de las redes actuales. Los operadores de red suelen ser tambien proveedores de servicios, cobrando tarifas bajas y planas, independientemente del servicio ofrecido. Se necesita una gestion de recursos precisa para optimizar su utilizacion, y para permitir nuevas sinergias entre operadores y proveedores de servicios. Concretamente, los operadores podrian abrir sus redes a terceros compartiendolas de forma flexible y personalizada para mejorar la calidad de servicio a cambio de aumentar sus ganancias como incentivo para mejorar sus infraestructuras. El objetivo principal de esta tesis es estudiar el potencial de los mecanismos de gestion y comparticion de recursos a pequei\a escala para trazar un camino sostenible hacia el 5G. En concreto, se estudian las arquitecturas y tecnolog fas mas avanzadas de "programabilidad" y escalabilidad de las redes, junto a un nuevo paradigma para la diversificacion de servicios y la comparticion de recursos. Revisamos los limites de las redes convencionales, ampliamos los esfuerzos de estandarizacion existentes y definimos una arquitectura para habilitar la centralizacion y la programabilidad en toda la red. La arquitectura propuesta se evalua en terminos de flexibilidad en la comparticion de recursos, y de mejora en la prestacion de servicios, mientras que las ventajas de un modelo de negocio alternativo se estudian en terminos de ganancia para los operadores. En primer lugar, estudiamos el aumento en la tasa de datos gracias a un uso compartido del espectro y de las infraestructuras, y evaluamos la mejora en las ganancias de los operadores. Presentamos un esquema de admision basado en la teoria de juegos para acomodar mas solicitudes de servicio cuando se adopta un enfoque cooperativo, y para estudiar las condiciones para que la reparticion de recursos sea conveniente entre coaliciones de operadores. Los resultados ensei\an que: i) la colaboracion puede ser favorable tambien en caso de una redistribucion desigual de los costes en cada coalicion; ii) las coaliciones de operadores de igual tamai\o ofrecen mejores ganancias y requieren tarifas mas bajas. El segundo tipo de comparticion que consideramos se da entre operadores de red y proveedores de servicios, en forma de provision de recursos personalizada ya pequei\a escala. Definimos un mecanismo de control de trafico basado en polfticas de admision, cuyo rendimiento se compara con estrategias de referencia. El mecanismo propuesto se basa en la teoria de subastas y calcula la politica de admision optima con una complejidad reducida para diferentes cargas de trafico y tasa de asignacion. Con particular atencion a servicios 5G de baja latencia, comparamos las prestaciones de estrategias convencionales para el control de admision con las del metodo propuesto, que proporciona: i) un suministro de servicios casi en tiempo real; ii) una complejidad reducida; iii) unos ingresos elevados; y iv) unos gastos reducidos, a cambio de unas perdidas insignificantes en terminos de imparcialidad hacia los proveedores de servicios. Para concluir, estudiamos el caso en el que se adoptan escalas de tiempo adaptables para el control de admision, con el fin de garantizar puntualmente los requisitos de servicio bajo diferentes condiciones de trafico. Para reducir la complejidad, consideramos el calculo previo de las estrategias de admision con respecto a condiciones de red de referenda, adaptables a condiciones inexploradas por medio de metodologias computacionalmente eficientes. Se compara el rendimiento de diferentes estrategias de admision sobre trazas de trafico real. Los resultados muestran que la estrategia propuesta equilibra complejidad y ganancias, mientras se reduce la utilizacion de recursos y la necesidad de conocer el estado exacto de la red.Postprint (published version

A Shared-Path Shared-Compute Planning Strategy for a Resilient Hybrid C-RAN

One key challenge in 5G networks is to guarantee the survivability of services in the event of failures. This paper focuses on the hybrid cloud radio access network (H-CRAN) architecture. The proposed strategy guarantees survivability in the presence of failures affecting nodes/links in the midhaul segment (i.e., connecting the radio aggregation unit (RAU) nodes to their respective radio cloud center (RCC) nodes) as well as compute resources (i.e., servers) in the RCC nodes. In the envisioned strategy each RAU node is connected to a primary and a backup RCC node (i.e., with backup compute resources) via two node disjoint connectivity paths in the midhaul. The proposed strategy, called Shared-Path Shared-Compute Planning (SPSCP), lowers the overall design cost by trying to share as much as possible backup connectivity and compute resources among RAU nodes. This is made possible by introducing a shareability metric early into the RCC node selection process so that the chance of sharing backup resources is maximized. Simulation results show that the SPSCP strategy can lead to up to 28% cost savings when compared to conventional resilient design strategie

Software Defined Applications in Cellular and Optical Networks

abstract: Small wireless cells have the potential to overcome bottlenecks in wireless access through the sharing of spectrum resources. A novel access backhaul network architecture based on a Smart Gateway (Sm-GW) between the small cell base stations, e.g., LTE eNBs, and the conventional backhaul gateways, e.g., LTE Servicing/Packet Gateways (S/P-GWs) has been introduced to address the bottleneck. The Sm-GW flexibly schedules uplink transmissions for the eNBs. Based on software defined networking (SDN) a management mechanism that allows multiple operator to flexibly inter-operate via multiple Sm-GWs with a multitude of small cells has been proposed. This dissertation also comprehensively survey the studies that examine the SDN paradigm in optical networks. Along with the PHY functional split improvements, the performance of Distributed Converged Cable Access Platform (DCCAP) in the cable architectures especially for the Remote-PHY and Remote-MACPHY nodes has been evaluated. In the PHY functional split, in addition to the re-use of infrastructure with a common FFT module for multiple technologies, a novel cross functional split interaction to cache the repetitive QAM symbols across time at the remote node to reduce the transmission rate requirement of the fronthaul link has been proposed.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201