Legacy Network Integration with SDN-IP Implementation towards a Multi-Domain SoDIP6 Network Environment

[EN] The logical separation of the data plane and the control plane of the network device conceptually defined by software-defined networking (SDN) creates many opportunities to create smart networking with better efficiency for network management and operation. SDN implementation over telecommunications (Telcos) and Internet service provider (ISP) networks is a challenging issue due to the lack of a high maturity level of SDN-based standards and several other critical factors that are considered during the real-time migration of existing legacy IPv4 networks. Different migration approaches have been studied; however, none of them seem to be close to realizing implementation. This paper implements the SDN-IP and Open Network Operating System (ONOS) SDN controller to migrate legacy IPv4 networks to multi-domain software-defined IPv6 (SoDIP6) networks and experimentally evaluate the viability of joint network migration in the ISP networks. We present results using extensive simulations for the suitable placement of the master ONOS controller during network migration by considering minimum control path latency using optimal path routing and the breadth first router replacement (BFR) technique. Our empirical analysis and evaluations show that the identification of the median router to attach the master controller and router migration planning using BFR give better results for carrier-grade legacy networks' migration to SoDIP6 networks.This research was partially funded by the Norwegian University of Science and Technology, Trondhiem, Norway (NTNU) under Sustainable Engineering Education Project (SEEP) financed by EnPE, University Grant Commission (grant-ID: FRG7475Engg01), Bhaktapur, Nepal, Nepal academy of Science and Technology (NAST), Kathmandu, Nepal, and U.S. National Science Foundation (NSF). The work of Danda B. Rawat was partly supported by the U.S. National Science Foundation (NSF) under grants CNS 1650831 and HRD 1828811. Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the NSF. We are thankful to the ERASMUS+ KA107 project and the GRC lab team members at Universitat Politècnica De València for the research support and facilitation.Dawadi, BR.; Rawat, DB.; Joshi, SR.; Manzoni, P. (2020). Legacy Network Integration with SDN-IP Implementation towards a Multi-Domain SoDIP6 Network Environment. Electronics. 9(9):1-22. https://doi.org/10.3390/electronics9091454S12299Dawadi, B. R., Rawat, D. B., & Joshi, S. R. (2019). Software Defined IPv6 Network: A New Paradigm for Future Networking. Journal of the Institute of Engineering, 15(2), 1-13. doi:10.3126/jie.v15i2.27636Dawadi, B. R., Rawat, D. B., Joshi, S. R., & Manzoni, P. (2020). Evolutionary gaming approach for decision making of Tier‐3 Internet service provider networks migration to SoDIP6 networks. International Journal of Communication Systems, 33(11). doi:10.1002/dac.4399Gu, D., Su, J., Xue, Y., Wang, D., Li, J., Luo, Z., & Yan, B. (2020). Modeling IPv6 adoption from biological evolution. Computer Communications, 158, 166-177. doi:10.1016/j.comcom.2020.02.081IPv6 Capability Measurement https://stats.labs.apnic.net/ipv6Dawadi, B. R., Rawat, D. B., Joshi, S. R., & Keitsch, M. M. (2018). Joint Cost Estimation Approach for Service Provider Legacy Network Migration to Unified Software Defined IPv6 Network. 2018 IEEE 4th International Conference on Collaboration and Internet Computing (CIC). doi:10.1109/cic.2018.00056Csikor, L., Szalay, M., Retvari, G., Pongracz, G., Pezaros, D. P., & Toka, L. (2020). Transition to SDN is HARMLESS: Hybrid Architecture for Migrating Legacy Ethernet Switches to SDN. IEEE/ACM Transactions on Networking, 28(1), 275-288. doi:10.1109/tnet.2019.2958762Sandhya, Sinha, Y., & Haribabu, K. (2017). A survey: Hybrid SDN. Journal of Network and Computer Applications, 100, 35-55. doi:10.1016/j.jnca.2017.10.003Mostafaei, H., Lospoto, G., Di Lallo, R., Rimondini, M., & Di Battista, G. (2020). A framework for multi‐provider virtual private networks in software‐defined federated networks. International Journal of Network Management, 30(6). doi:10.1002/nem.2116Dawadi, B. R., Rawat, D. B., & Joshi, S. R. (2019). Evolutionary Dynamics of Service Provider Legacy Network Migration to Software Defined IPv6 Network. Advances in Intelligent Systems and Computing, 245-257. doi:10.1007/978-3-030-19861-9_24Salsano, S., Ventre, P. L., Lombardo, F., Siracusano, G., Gerola, M., Salvadori, E., … Prete, L. (2016). Hybrid IP/SDN Networking: Open Implementation and Experiment Management Tools. IEEE Transactions on Network and Service Management, 13(1), 138-153. doi:10.1109/tnsm.2015.2507622Vissicchio, S., Tilmans, O., Vanbever, L., & Rexford, J. (2015). Central Control Over Distributed Routing. Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. doi:10.1145/2785956.2787497Rizvi, S. N., Raumer, D., Wohlfart, F., & Carle, G. (2015). Towards carrier grade SDNs. Computer Networks, 92, 218-226. doi:10.1016/j.comnet.2015.09.029Risdianto, A. C., Tsai, P.-W., Ling, T. C., Yang, C.-S., & Kim, J. (2017). Enhanced Onos Sdn Controllers Deployment For Federated Multi-Domain Sdn-Cloud With Sd-Routing-Exchange. Malaysian Journal of Computer Science, 30(2), 134-153. doi:10.22452/mjcs.vol30no2.5Ventre, P. L., Salsano, S., Gerola, M., Salvadori, E., Usman, M., Buscaglione, S., … Snow, W. (2017). SDN-Based IP and Layer 2 Services with an Open Networking Operating System in the GÉANT Service Provider Network. IEEE Communications Magazine, 55(4), 71-79. doi:10.1109/mcom.2017.1600194SDN-IP Arhitecture https://wiki.onosproject.org/display/ONOS/SDN-IP+ArchitectureLee, H.-L., Liu, T.-L., & Chen, M. (2019). Deploying SDN-IP over Transnational Network Testbed. 2019 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW). doi:10.1109/icce-tw46550.2019.8991776Das, T., Sridharan, V., & Gurusamy, M. (2020). A Survey on Controller Placement in SDN. IEEE Communications Surveys & Tutorials, 22(1), 472-503. doi:10.1109/comst.2019.2935453Chen, W., Chen, C., Jiang, X., & Liu, L. (2018). Multi-Controller Placement Towards SDN Based on Louvain Heuristic Algorithm. IEEE Access, 6, 49486-49497. doi:10.1109/access.2018.2867931Qi, Y., Wang, D., Yao, W., Li, H., & Cao, Y. (2019). Towards Multi-Controller Placement for SDN Based on Density Peaks Clustering. ICC 2019 - 2019 IEEE International Conference on Communications (ICC). doi:10.1109/icc.2019.8761814Lu, J., Zhang, Z., Hu, T., Yi, P., & Lan, J. (2019). A Survey of Controller Placement Problem in Software-Defined Networking. IEEE Access, 7, 24290-24307. doi:10.1109/access.2019.2893283Singh, A. K., Maurya, S., Kumar, N., & Srivastava, S. (2019). Heuristic approaches for the reliable SDN controller placement problem. Transactions on Emerging Telecommunications Technologies, 31(2). doi:10.1002/ett.3761Das, T., & Gurusamy, M. (2018). Resilient Controller Placement in Hybrid SDN/Legacy Networks. 2018 IEEE Global Communications Conference (GLOBECOM). doi:10.1109/glocom.2018.8647566Heller, B., Sherwood, R., & McKeown, N. (2012). The controller placement problem. ACM SIGCOMM Computer Communication Review, 42(4), 473-478. doi:10.1145/2377677.2377767SDN Control Plane Performance: Raising the Bar on SDN Performance, Scalability, and High Availability https://wiki.onosproject.org/download/attachments/13994369/Whitepaper-%20ONOS%20Kingfisher%20release%20performance.pdf?version=

A balanced partitioning mechanism for multicontroller placement in software-defined wide area networks

Through softwarization, Software-Defined Networking (SDN) may govern the network. Deploying a single controller to manage enormous network traffic is inefficient; hence, having multiple controllers is a necessity of current SDN in wide area networks (WANs). However, the controller placement problem (CPP) is a thriving research subject for efficiently placing many controllers to improve network performance. It has two parts: how the controllers should be distributed and how many networking devices each controller should be connected to. Consequently, the objective of this study is to propose a Balanced Partitioning Mechanism (BPM) based on the notion of a network partition. Moreover, the BPM is designed based on a modified K-means algorithm. BPM comprises of two approaches: the initialization method and the partitioning strategy. The farthest-point initialization method is introduced to reduce end-to-end delay between the controllers and switches. The balanced partitioning strategy is used to balance controller loads and partition the network into balanced partitions. The research adopted the Design Science Research Methodology (DSRM) to accomplish its objectives. The network simulator OMNeT++ was configured to simulate the performance of BPM over the OS3E topology, with two scenarios including five and six domains. The K-means and CNPA algorithms, in particular, were used to evaluate the performance of BPM. In terms of balanced partitioning, the findings reveal that BPM outperforms the K-means and CNPA algorithms by maintaining a good load balance among controllers. Furthermore, the results show that BPM improves throughput and reduces end-to-end delay between the controllers and switches. In addition, BPM improves the number of packets received by the destination to the number of packets sent by 23% and 29% compared to the K-means for five and six domain scenarios, respectively. Given the diversity of future Internet and IoT, the findings have significant implications for improving the performance of WAN networks

Scalable ReliableControllerPlacementinSoftwareDefinedNetworking

Software Defined Networking (SDN) is a new networking paradigm that facilitates a centralized system of computer networks by decoupling the control and data plane from each other, where a controller maintains the management of a global view of the network. SDN architectures can provide programmatic interfaces in communication networks that significantly simplify network management. Hence, the controllability and manageability of a network can be improved. On the one hand, the placement of controllers can significantly impact network performance in terms of controller responsiveness. On the other hand, SDN offers the ability to have controllers distributed over the network to solve the single point of failure problem at the control plane, increasing scalability and flexibility. However, there are some inevitable problems for such networks, especially for controller-related problems. For instance, scalability, reliability, and controller availability are some of the hottest aspects of SDN. More precisely, failure of the controllers themselves may lead to the impact of these aspects and the collapse of the network performance. Despite the issues mentioned above, the controller placement challenges must be appropriately addressed to take advantage of the SDN. The connections between the controller (control plane) and the switches (data plane) in SDN are established by either an in-band or an out-of-band control mechanism. New challenges still arise regardin the connection availability and provide more protection for the connection between the data and control planes. A disconnection between the two planes could result in performance degradation. Although the SDN offers the advantage of an environment of multiple distributed controllers, yet the intercommunication factor between these controllers is still a key challenge. This thesis investigates the issues mentioned above and organizes them into four stages. First, dealing with the controller placement problem as the most crucial concern in SDN, via exploiting the independent dominating set approach to ensure a distribution of controllers with lowest response times. We propose a new node degree-based algorithm named High Degree with Independent Dominating Set (HDIDS) for the controller placement problem in the SDN networks. HDIDS is composed of two phases to deal with controller placement: (1) determining candidate controller instances by selecting those nodes with the highest degree; and (2) partitioning the network into multiple domains, one controller per domain. To further improve network performance, reliability, and survivability, one solution is to deploy backup controllers to satisfy the quality of service requirements. In this regard, as a second step, we enhance the controller placement approach by designing a reliable and survivable controller placement strategy. This strategy relies on the efficient deployment of backup controllers by constructing virtual backup domains set(s) to ensure the durability and resilience of network control management. The approach design is called a Survivable Backup Controller Placement approach. Furthermore, to achieve reliable control traffic between data and control planes in an in-band control network, as a third stage, we design and implement an In-band Control Protection Module that finds a set of ideal paths for the control channel under the failure conditions. The proposed protection mechanism protects as much control traffic as possible. Finally, we present a practical approach for the controller placement problem in software defined networks aiming to minimize the inter-controller communication delay time and the delay time between controller and switches. The principal concept employed in this approach is the Connected Dominating Set. Further, we present an algorithm using the Minimum Connected Dominating Set, which minimizes the delay time between the distributed SDN controllers

Performance Evaluation of the Control Plane in OpenFlow Networks

Online services and applications have grown rapidly in the last decade. The network is necessary for many services and applications. Many technologies are invented to meet the requirements of online services, such as micro-services and serverless computing. However, the traditional network architecture suffers from several shortages. It is difficult for the traditional network to adapt to new demands without massive reconfiguration. In traditional IP networks, it is complex to manage and configure the network devices since skilled technicians are required. Changing the policy of a network is also time consuming because network operators need to re-configure multiple network devices and update access control lists using low level commands. The management and configuration becomes more complex and challenging, when the traffic in a network changes frequently. SDN (Software-defined networking) is an innovative approach to manage networks more flexible. It separates the control plane from forwarding devices and uses a centralized controller to manipulate all the forwarding devices. The separation offers many benefits in terms of network flexibility and management. The controller can provide a global view of a network. Using the controller, network operators can manage and configure all the network devices at a high level interface. With SDN, a network can adapt to new demands by updating the applications in the controller. However, all these benefits come with a performance penalty. Since the controller manipulates all the forwarding devices, the performance of the controller impacts the performance of the whole network. In this thesis, we investigate the performance of SDN controllers. We also implement a benchmark tool for OpenFlow controllers. It measures the response time of an OpenFlow controller and fit a phase-type distribution to the response time. Based on the distribution of the response time, we build a queueing model for multiple controllers in an OpenFlow network and determine the optimal number of controllers that can minimize the response time of the controllers. We design an algorithm that can optimize the mapping relationship among the switches and controllers. The load of controllers can be balanced with the optimized mapping relationship

Assurance, Provision, Management and Enhancement of QoS in 5G Communication Networks

Enhancement of QoS in PS network as 5G communication network is non trivial endeavour which faces a host of new challenges beyond 3G and 4G communication networks. The number of nodes, the homogeneity of the access technologies, the conflicting network management objectives, resource usage minimization, and the division between limited physical resources and elastic virtual resources is driving a complete change in the vision and methodologies for efficient management of the available network resources. QoS is the measure of the reliability and performance of the networks’ nodes and links, particularly as perceivedbytheendusersoftheservicesandapplicationthataretransportedviaPSnetwork. Furthermore, QoS is a composite metric as it based on a number of multiple factors, which indicate the E2E characteristics and performance of the network condition, applications and services. Hence, reductions or improvements in the QoS level can brought about through a number of combined factors. This thesis tries to introduce a vision of Quality of Service (QoS) enhancement and management based on the 5th generation network requirements and solutions by: Firstly: Proposing a traffic flow management policy, which allocates and organises Machine Type Communication (MTC) traffic flow’s network resources sharing within Evolved Packet System (EPS), with an access element as a Wireless Sensor Network (WSN) gateway for providing an overlaying access channel between the Machine Type Devices (MTDs) and EPS. This proposal addresses the effect and interaction in the heterogeneity of applications, services and terminal devices and the related QoS issues among them. The introduced work inthisproposalovercomestheproblemsofnetworkresourcestarvationbypreventingdeterioration of network performance. The scheme is validated through simulation, which indicates the proposed traffic flow management policy outperforms the current traffic management policy. Specifically, simulation results show that the proposed model achieves an enhancement in QoS performance for the MTC traffic flows, including a decrease of 99.45% in Packet Loss Rate (PLR), a decrease of 99.89% in packet End to End (E2E) delay, a decrease of 99.21% in Packet Delay Variation (PDV). Furthermore, it retains the perceived Quality of Experience (QoE) of the real time application users within high satisfaction levels, such as the Voice over Long Term Evolution (VoLTE) service possessing a Mean Opinion Score (MOS)of4.349andenhancingtheQoSofavideoconferenceservicewithinthestandardised values of a 3GPP body, with a decrease of 85.28% in PLR, a decrease of 85% in packet E2E delay and a decrease of 88.5% in PDV. Secondly: Proposing an approach for allocating existing 4G installed network radio access nodes to multiple Base Band Unit (BBU) pools, which is proposed to deploy 5G Cloud-Radio Access Network (C-RAN) and improve the offered Network QoS (NQoS). The proposed approach involves performing radio access nodes clustering based on the Particle Swarm Optimization (PSO) algorithm, model selection Bayesian Information Criterion (BIC), Measure of spread technique and Voronoi tessellation. The proposed scheme is used to consider a Dynamic C-RAN (DC-RAN) operation, that adaptively adjusts the main Radio Remote Head (RRH) coverage range according to the traffic load requirement as well as considering energy saving. The numerical results of the approach show that the optimized partition of the proposed network model is 41 BBU pools, with an average density of RRHs per pool area, which matches the primary average density of the radio access nodes per network area. Thirdly: Developing mathematical framework that investigates the Power Consumption (PC) profile for the interaction of Internet of Thing (IoT) Application QoS (AQoS) with NQoS in wireless Software Defined Network (SDN) as SDN for WIreless SEnsor network (SDN-WISE). This profile model offers flexibility for managing the structure of the Machine to Machine (M2M) system in IoT. It enables controlling the provided NQoS, precisely the achieved PHY layer transmission link throughput, combined with the AQoS, represented by IoT data stream payload size. The investigation is composed of two essential SDN traffic parts, they are control plane signalling and data plane traffic PCs and their relevance with QoS. The results show that 98% PC in data plane companion with a control plane PC of 2% in overall of the proposed system power, these figures were achieved with control plane signalling Transmission Time Interval (TTI) of 5 sec and a maximum data plane payload size of 92 Bytes as a worst case scenario.Ministry of Higher Education and Scientific Research (MOHESR), Cultural Attache and University of Wasit in Ira

Study and application of spectral monitoring techniques for optical network optimization

One of the possible ways to address the constantly increasing amount of heterogeneous and variable internet traffic is the evolution of the current optical networks towards a more flexible, open, and disaggregated paradigm. In such scenarios, the role played by Optical Performance Monitoring (OPM) is fundamental. In fact, OPM allows to balance performance and specification mismatches resulting from the disaggregation adoption and provides the control plane with the necessary feedback to grant the optical networks an adequate automation level. Therefore, new flexible and cost-effective OPM solutions are needed, as well as novel techniques to extract the desired information from the monitored data and process and apply them. In this dissertation, we focus on three aspects related to OPM. We first study a monitoring data plane scheme to acquire the high resolution signal optical spectra in a nonintrusive way. In particular, we propose a coherent detection based Optical Spectrum Analyzer (OSA) enhanced with specific Digital Signal Processing (DSP) to detect spectral slices of the considered optical signals. Then, we identify two main placement strategies for such monitoring solutions, enhancing them using two spectral processing techniques to estimate signal- and optical filter-related parameters. Specifically, we propose a way to estimate the Amplified Spontaneous Emission (ASE) noise or its related Optical Signal-to-Noise (OSNR) using optical spectra acquired at the egress ports of the network nodes and the filter central frequency and 3/6 dB bandwidth, using spectra captured at the ingress ports of the network nodes. To do so, we leverage Machine Learning (ML) algorithms and the function fitting principle, according to the considered scenario. We validate both the monitoring strategies and their related processing techniques through simulations and experiments. The obtained results confirm the validity of the two proposed estimation approaches. In particular, we are able to estimate in-band the OSNR/ASE noise within an egress monitor placement scenario, with a Maximum Absolute Error (MAE) lower than 0.4 dB. Moreover, we are able to estimate the filter central frequency and 3/6 dB bandwidth, within an ingress optical monitor placement scenario, with a MAE lower than 0.5 GHz and 0.98 GHz, respectively. Based on such evaluations, we also compare the two placement scenarios and provide guidelines on their implementation. According to the analysis of specific figures of merit, such as the estimation of the Signal-to-Noise Ratio (SNR) penalty introduced by an optical filter, we identify the ingress monitoring strategy as the most promising. In fact, when compared to scenarios where no monitoring strategy is adopted, the ingress one reduced the SNR penalty estimation by 92%. Finally, we identify a potential application for the monitored information. Specifically, we propose a solution for the optimization of the subchannel spectral spacing in a superchannel. Leveraging convex optimization methods, we implement a closed control loop process for the dynamical reconfiguration of the subchannel central frequencies to optimize specific Quality of Transmission (QoT)-related metrics. Such a solution is based on the information monitored at the superchannel receiver side. In particular, to make all the subchannels feasible, we consider the maximization of the total superchannel capacity and the maximization of the minimum superchannel subchannel SNR value. We validate the proposed approach using simulations, assuming scenarios with different subchannel numbers, signal characteristics, and starting frequency values. The obtained results confirm the effectiveness of our solution. Specifically, compared with the equally spaced subchannel scenario, we are able to improve the total and the minimum subchannel SNR values of a four subchannel superchannel, of 1.45 dB and 1.19 dB, respectively.Una de las posibles formas de hacer frente a la creciente cantidad de tráfico heterogéneo y variable de Internet es la evolución de las actuales redes ópticas hacia un paradigma más flexible, abierto y desagregado. En estos escenarios, el papel que desempeña el modulo óptico de monitorización de prestaciones (OPM) es fundamental. De hecho, el OPM permite equilibrar los desajustes de rendimiento y especificación, los cuales surgen con la adopción de la desagregación; del mismo modo el OPM también proporciona al plano de control la realimentación necesaria para otorgar un nivel de automatización adecuado a las redes ópticas. En esta tesis, nos centramos en tres aspectos relacionados con el OPM. En primer lugar, estudiamos un esquema de monitorización para adquirir, de forma no intrusiva, los espectros ópticos de señales de alta resolución. En concreto, proponemos un analizador de espectro óptico (OSA) basado en detección coherente y mejorado con un específico procesado digital de señal (DSP) para detectar cortes espectrales de las señales ópticas consideradas. A continuación, presentamos dos técnicas de colocación para dichas soluciones de monitorización, mejorándolas mediante dos técnicas de procesamiento espectral para estimar los parámetros relacionados con la señal y el filtro óptico. Específicamente, proponemos un método para estimar el ruido de emisión espontánea amplificada (ASE), o la relación de señal-ruido óptica (OSNR), utilizando espectros ópticos adquiridos en los puertos de salida de los nodos de la red. Del mismo modo, estimamos la frecuencia central del filtro y el ancho de banda de 3/6 dB, utilizando espectros capturados en los puertos de entrada de los nodos de la red. Para ello, aprovechamos los algoritmos de Machine Learning (ML) y el principio de function fitting, según el escenario considerado. Validamos tanto las estrategias de monitorización como las técnicas de procesamiento mediante simulaciones y experimentos. Se puede estimar en banda el ruido ASE/OSNR en un escenario de colocación de monitores de salida, con un Maximum Absolute Error (MAE) inferior a 0.4 dB. Además, se puede estimar la frecuencia central del filtro y el ancho de banda de 3/6 dB, dentro de un escenario de colocación de monitores ópticos de entrada, con un MAE inferior a 0.5 GHz y 0.98 GHz, respectivamente. A partir de estas evaluaciones, también comparamos los dos escenarios de colocación y proporcionamos directrices sobre su aplicación. Según el análisis de específicas figuras de mérito, como la estimación de la penalización de la relación señal-ruido (SNR) introducida por un filtro óptico, demostramos que la estrategia de monitorización de entrada es la más prometedora. De hecho, utilizar un sistema de monitorización de entrada redujo la estimación de la penalización del SNR en un 92%. Por último, identificamos una posible aplicación para la información monitorizada. En concreto, proponemos una solución para la optimización del espaciado espectral de los subcanales en un supercanal. Aprovechando los métodos de optimización convexa, implementamos un proceso cíclico de control cerrado para la reconfiguración dinámica de las frecuencias centrales de los subcanales con el fin de optimizar métricas específicas relacionadas con la calidad de la transmisión (QoT). Esta solución se basa en la información monitorizada en el lado del receptor del supercanal. Validamos el enfoque propuesto mediante simulaciones, asumiendo escenarios con un diferente número de subcanales, distintas características de la señal, y diversos valores de la frecuencia inicial. Los resultados obtenidos confirman la eficacia de nuestra solución. Más específicatamente, en comparación con el escenario de subcanales igualmente espaciados, se pueden mejorar los valores totales y minimos de SNR de los subcanales de un supercanal de cuatro subcanales, de 1.45 dB y 1.19 dB, respectivamentePostprint (published version

Traffic control for energy harvesting virtual small cells via reinforcement learning

Due to the rapid growth of mobile data traffic, future mobile networks are expected to support at least 1000 times more capacity than 4G systems. This trend leads to an increasing energy demand from mobile networks which raises both economic and environmental concerns. Energy costs are becoming an important part of OPEX by Mobile Network Operators (MNOs). As a result, the shift towards energy-oriented design and operation of 5G and beyond systems has been emphasized by academia, industries as well as standard bodies. In particular, Radio Access Network (RAN) is the major energy consuming part of cellular networks. To increase the RAN efficiency, Cloud Radio Access Network (CRAN) has been proposed to enable centralized cloud processing of baseband functions while Base Stations (BSs) are reduced to simple Radio Remote Heads (RRHs). The connection between the RRHs and central cloud is provided by high capacity and very low latency fronthaul. Flexible functional splits between local BS sites and a central cloud are then proposed to relax the CRAN fronthaul requirements via partial processing of baseband functions at the local BS sites. Moreover, Network Function Virtualization (NFV) and Software Defined Networking (SDN) enable flexibility in placement and control of network functions. Relying on SDN/NFV with flexible functional splits, network functions of small BSs can be virtualized and placed at different sites of the network. These small BSs are known as virtual Small Cells (vSCs). More recently, Multi-access Edge Computing (MEC) has been introduced where BSs can leverage cloud computing capabilities and offer computational resources on demand basis. On the other hand, Energy Harvesting (EH) is a promising technology ensuring both cost effectiveness and carbon footprint reduction. However, EH comes with challenges mainly due to intermittent and unreliable energy sources. In EH Base Stations (EHBSs), it is important to intelligently manage the harvested energy as well as to ensure energy storage provision. Consequently, MEC enabled EHBSs can open a new frontier in energy-aware processing and sharing of processing units according to flexible functional split options. The goal of this PhD thesis is to propose energy-aware control algorithms in EH powered vSCs for efficient utilization of harvested energy and lowering the grid energy consumption of RAN, which is the most power consuming part of the network. We leverage on virtualization and MEC technologies for dynamic provision of computational resources according to functional split options employed by the vSCs. After describing the state-of-the-art, the first part of the thesis focuses on offline optimization for efficient harvested energy utilization via dynamic functional split control in vSCs powered by EH. For this purpose, dynamic programming is applied to determine the performance bound and comparison is drawn against static configurations. The second part of the thesis focuses on online control methods where reinforcement learning based controllers are designed and evaluated. In particular, more focus is given towards the design of multi-agent reinforcement learning to overcome the limitations of centralized approaches due to complexity and scalability. Both tabular and deep reinforcement learning algorithms are tailored in a distributed architecture with emphasis on enabling coordination among the agents. Policy comparison among the online controllers and against the offline bound as well as energy and cost saving benefits are also analyzed.Debido al rápido crecimiento del tráfico de datos móviles, se espera que las redes móviles futuras admitan al menos 1000 veces más capacidad que los sistemas 4G. Esta tendencia lleva a una creciente demanda de energía de las redes móviles, lo que plantea preocupaciones económicas y ambientales. Los costos de energía se están convirtiendo en una parte importante de OPEX por parte de los operadores de redes móviles (MNO). Como resultado, la academia, las industrias y los organismos estándar han enfatizado el cambio hacia el diseño orientado a la energía y la operación de sistemas 5G y más allá de los sistemas. En particular, la red de acceso por radio (RAN) es la principal parte de las redes celulares que consume energía. Para aumentar la eficiencia de la RAN, se ha propuesto Cloud Radio Access Network (CRAN) para permitir el procesamiento centralizado en la nube de las funciones de banda base, mientras que las estaciones base (BS) se reducen a simples cabezales remotos de radio (RRH). La conexión entre los RRHs y la nube central es proporcionada por una capacidad frontal de muy alta latencia y muy baja latencia. Luego se proponen divisiones funcionales flexibles entre los sitios de BS locales y una nube central para relajar los requisitos de red de enlace CRAN a través del procesamiento parcial de las funciones de banda base en los sitios de BS locales. Además, la virtualización de funciones de red (NFV) y las redes definidas por software (SDN) permiten flexibilidad en la colocación y el control de las funciones de red. Confiando en SDN / NFV con divisiones funcionales flexibles, las funciones de red de pequeñas BS pueden virtualizarse y ubicarse en diferentes sitios de la red. Estas pequeñas BS se conocen como pequeñas celdas virtuales (vSC). Más recientemente, se introdujo la computación perimetral de acceso múltiple (MEC) donde los BS pueden aprovechar las capacidades de computación en la nube y ofrecer recursos computacionales según la demanda. Por otro lado, Energy Harvesting (EH) es una tecnología prometedora que garantiza tanto la rentabilidad como la reducción de la huella de carbono. Sin embargo, EH presenta desafíos principalmente debido a fuentes de energía intermitentes y poco confiables. En las estaciones base EH (EHBS), es importante administrar de manera inteligente la energía cosechada, así como garantizar el suministro de almacenamiento de energía. En consecuencia, los EHBS habilitados para MEC pueden abrir una nueva frontera en el procesamiento con conciencia energética y el intercambio de unidades de procesamiento de acuerdo con las opciones de división funcional flexible. El objetivo de esta tesis doctoral es proponer algoritmos de control conscientes de la energía en vSC alimentados por EH para la utilización eficiente de la energía cosechada y reducir el consumo de energía de la red de RAN, que es la parte más consumidora de la red. Aprovechamos las tecnologías de virtualización y MEC para la provisión dinámica de recursos computacionales de acuerdo con las opciones de división funcional empleadas por los vSC. La primera parte de la tesis se centra en la optimización fuera de línea para la utilización eficiente de la energía cosechada a través del control dinámico de división funcional en vSC con tecnología EH. Para este propósito, la programación dinámica se aplica para determinar el rendimiento limitado y la comparación se realiza con configuraciones estáticas. La segunda parte de la tesis se centra en los métodos de control en línea donde se diseñan y evalúan los controladores basados en el aprendizaje por refuerzo. En particular, se presta más atención al diseño de aprendizaje de refuerzo de múltiples agentes para superar las limitaciones de los enfoques centralizados debido a la complejidad y la escalabilidad. También se analiza la comparación de políticas entre los controladores en línea y contra los límites fuera de línea,Postprint (published version