DeepWiN: Deep Graph Reinforcement Learning for User-Centric Radio Access Networks Automation

The future cellular networks are expected to support an increasing number of users with heterogeneous applications, requiring varying network resources. Therefore, the 6G and beyond cellular networks need to be elastic, and user-centric. User-centric Radio Access Networks (UCRAN), with virtual cells (S-zones), can provide on-demand connectivity, coverage and quality of service to different user applications while optimizing the network for energy efficiency, area spectral efficiency, reliability and user service rate. However, with high variability in the network, due to user mobility and fading, the selection of S-zone sizes which optimize the network performance for multiple types of users simultaneously becomes a challenge. Therefore, to automate the selection of S-zone sizes dynamically, we propose deep graph reinforcement learning (DGRL), a Soft actor-critic model integrated with Graph neural network. DGRL infers from DeepWiN, a graphical representation of UCRAN that encodes the non-euclidean topology of the network along with its euclidean features, effectively encapsulating the wireless domain knowledge of the network configuration. Our experiments show that the deep graph reinforcement learning can learn to optimize S-zone sizes with 15% fewer training episodes in comparison to the legacy neural-network-based reinforcement learning, hence demonstrating the advantage of network topology-awareness for artificial intelligence

Network resource allocation policies with energy transfer capabilities

During the last decades, mobile network operators have witnessed an exponential increase in the traffic demand, mainly due to the high request of services from a huge amount of users. The trend is of a further increase in both the traffic demand and the number of connected devices over the next years. The traffic load is expected to have an annual growth rate of 53% for the mobile network alone, and the upcoming industrial era, which will connect different types of devices to the mobile infrastructure including human and machine type communications, will definitely exacerbate such an increasing trend. The current directions anticipate that future mobile networks will be composed of ultra dense deployments of heterogeneous Base Stations (BSs), where BSs using different transmission powers coexist. Accordingly, the traditional Macro BSs layer will be complemented or replaced with multiple overlapping tiers of small BSs (SBSs), which will allow extending the system capacity. However, the massive use of Information and Communication Technology (ICT) and the dense deployment of network elements is going to increase the level of energy consumed by the telecommunication infrastructure and its carbon footprint on the environment. Current estimations indicates that 10% of the worldwide electricity generation is due to the ICT industry and this value is forecasted to reach 51% by 2030, which imply that 23% of the carbon footprint by human activity will be due to ICT. Environmental sustainability is thus a key requirement for designing next generation mobile networks. Recently, the use of Renewable Energy Sources (RESs) for supplying network elements has attracted the attention of the research community, where the interest is driven by the increased efficiency and the reduced costs of energy harvesters and storage devices, specially when installed to supply SBSs. Such a solution has been demonstrated to be environmentally and economically sustainable in both rural and urban areas. However, RESs will entail a higher management complexity. In fact, environmental energy is inherently erratic and intermittent, which may cause a fluctuating energy inflow and produce service outage. A proper control of how the energy is drained and balanced across network elements is therefore necessary for a self-sustainable network design. In this dissertation, we focus on energy harvested through solar panels that is deemed the most appropriate due to the good efficiency of commercial photovoltaic panels as well as the wide availability of the solar source for typical installations. The characteristics of this energy source are analyzed in the first technical part of the dissertation, by considering an approach based on the extraction of features from collected data of solar energy radiation. In the second technical part of the thesis we introduce our proposed scenario. A federation of BSs together with the distributed harvesters and storage devices at the SBS sites form a micro-grid, whose operations are managed by an energy management system in charge of controlling the intermittent and erratic energy budget from the RESs. We consider load control (i.e., enabling sleep mode in the SBSs) as a method to properly manage energy inflow and spending, based on the traffic demand. Moreover, in the third technical part, we introduce the possibility of improving the network energy efficiency by sharing the exceeding energy that may be available at some BS sites within the micro-grid. Finally, a centralized controller based on supervised and reinforcement learning is proposed in the last technical part of the dissertation. The controller is in charge of opportunistically operating the network to achieve efficient utilization of the harvested energy and prevent SBSs blackout.Durante las últimas décadas, los operadores de redes móviles han sido testigos de un aumento exponencial en la demanda de tráfico, principalmente debido a la gran solicitud de servicios de una gran cantidad de usuarios. La tendencia es un aumento adicional tanto en la demanda de tráfico como en la cantidad de dispositivos conectados en los próximos años. Se espera que la carga de tráfico tenga una tasa de crecimiento anual del 53% solo para la red móvil, y la próxima era industrial, que conectará diferentes tipos de dispositivos a la infraestructura móvil, definitivamente exacerbará tal aumento. Las instrucciones actuales anticipan que las redes móviles futuras estarán compuestas por despliegues ultra densos de estaciones base (BS) heterogéneas. En consecuencia, la capa tradicional de Macro BS se complementará o reemplazará con múltiples niveles superpuestos de pequeños BS (SBS), lo que permitirá ampliar la capacidad del sistema. Sin embargo, el uso masivo de la Tecnología de la Información y la Comunicación (TIC) y el despliegue denso de los elementos de la red aumentará el nivel de energía consumida por la infraestructura de telecomunicaciones y su huella de carbono en el medio ambiente. Las estimaciones actuales indican que el 10% de la generación mundial de electricidad se debe a la industria de las TIC y se prevé que este valor alcance el 51% para 2030, lo que implica que el 23% de la huella de carbono por actividad humana se deberá a las TIC. La sostenibilidad ambiental es, por lo tanto, un requisito clave para diseñar redes móviles de próxima generación. Recientemente, el uso de fuentes de energía renovables (RES) para suministrar elementos de red ha atraído la atención de la comunidad investigadora, donde el interés se ve impulsado por el aumento de la eficiencia y la reducción de los costos de los recolectores y dispositivos de almacenamiento de energía, especialmente cuando se instalan para suministrar SBS. Se ha demostrado que dicha solución es ambiental y económicamente sostenible tanto en áreas rurales como urbanas. Sin embargo, las RES conllevarán una mayor complejidad de gestión. De hecho, la energía ambiental es inherentemente errática e intermitente, lo que puede causar una entrada de energía fluctuante y producir una interrupción del servicio. Por lo tanto, es necesario un control adecuado de cómo se drena y equilibra la energía entre los elementos de la red para un diseño de red autosostenible. En esta disertación, nos enfocamos en la energía cosechada a través de paneles solares que se considera la más apropiada debido a la buena eficiencia de los paneles fotovoltaicos comerciales, así como a la amplia disponibilidad de la fuente solar para instalaciones típicas. Las características de esta fuente de energía se analizan en la primera parte técnica de la disertación, al considerar un enfoque basado en la extracción de características de los datos recopilados de radiación de energía solar. En la segunda parte técnica de la tesis presentamos nuestro escenario propuesto. Una federación de BS junto con los cosechadores distribuidos y los dispositivos de almacenamiento forman una microrred, cuyas operaciones son administradas por un sistema de administración de energía a cargo de controlar el presupuesto de energía intermitente y errático de las RES. Consideramos el control de carga como un método para administrar adecuadamente la entrada y el gasto de energía, en función de la demanda de tráfico. Además, en la tercera parte técnica, presentamos la posibilidad de mejorar la eficiencia energética de la red al compartir la energía excedente que puede estar disponible en algunos sitios dentro de la microrred. Finalmente, se propone un controlador centralizado basado en aprendizaje supervisado y de refuerzo en la última parte técnica de la disertación. El controlador está a cargo de operar la red para lograr una utilización eficiente de energía y previene el apagón de SB

Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review

The field of Tiny Machine Learning (TinyML) has gained significant attention due to its potential to enable intelligent applications on resource-constrained devices. This review provides an in-depth analysis of the advancements in efficient neural networks and the deployment of deep learning models on ultra-low power microcontrollers (MCUs) for TinyML applications. It begins by introducing neural networks and discussing their architectures and resource requirements. It then explores MEMS-based applications on ultra-low power MCUs, highlighting their potential for enabling TinyML on resource-constrained devices. The core of the review centres on efficient neural networks for TinyML. It covers techniques such as model compression, quantization, and low-rank factorization, which optimize neural network architectures for minimal resource utilization on MCUs. The paper then delves into the deployment of deep learning models on ultra-low power MCUs, addressing challenges such as limited computational capabilities and memory resources. Techniques like model pruning, hardware acceleration, and algorithm-architecture co-design are discussed as strategies to enable efficient deployment. Lastly, the review provides an overview of current limitations in the field, including the trade-off between model complexity and resource constraints. Overall, this review paper presents a comprehensive analysis of efficient neural networks and deployment strategies for TinyML on ultra-low-power MCUs. It identifies future research directions for unlocking the full potential of TinyML applications on resource-constrained devices.Comment: 39 pages, 9 figures, 5 table

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

High-dimensional regression adjustments in randomized experiments

We study the problem of treatment effect estimation in randomized experiments with high-dimensional covariate information, and show that essentially any risk-consistent regression adjustment can be used to obtain efficient estimates of the average treatment effect. Our results considerably extend the range of settings where high-dimensional regression adjustments are guaranteed to provide valid inference about the population average treatment effect. We then propose cross-estimation, a simple method for obtaining finite-sample-unbiased treatment effect estimates that leverages high-dimensional regression adjustments. Our method can be used when the regression model is estimated using the lasso, the elastic net, subset selection, etc. Finally, we extend our analysis to allow for adaptive specification search via cross-validation, and flexible non-parametric regression adjustments with machine learning methods such as random forests or neural networks.Comment: To appear in the Proceedings of the National Academy of Sciences. The present draft does not reflect final copyediting by the PNAS staf