Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning

Multicasting in wireless systems is a natural way to exploit the redundancy in user requests in a Content Centric Network. Power control and optimal scheduling can significantly improve the wireless multicast network's performance under fading. However, the model based approaches for power control and scheduling studied earlier are not scalable to large state space or changing system dynamics. In this paper, we use deep reinforcement learning where we use function approximation of the Q-function via a deep neural network to obtain a power control policy that matches the optimal policy for a small network. We show that power control policy can be learnt for reasonably large systems via this approach. Further we use multi-timescale stochastic optimization to maintain the average power constraint. We demonstrate that a slight modification of the learning algorithm allows tracking of time varying system statistics. Finally, we extend the multi-timescale approach to simultaneously learn the optimal queueing strategy along with power control. We demonstrate scalability, tracking and cross layer optimization capabilities of our algorithms via simulations. The proposed multi-timescale approach can be used in general large state space dynamical systems with multiple objectives and constraints, and may be of independent interest.Comment: arXiv admin note: substantial text overlap with arXiv:1910.0530

Optimization and Performance Analysis of High Speed Mobile Access Networks

The end-to-end performance evaluation of high speed broadband mobile access networks is the main focus of this work. Novel transport network adaptive flow control and enhanced congestion control algorithms are proposed, implemented, tested and validated using a comprehensive High speed packet Access (HSPA) system simulator. The simulation analysis confirms that the aforementioned algorithms are able to provide reliable and guaranteed services for both network operators and end users cost-effectively. Further, two novel analytical models one for congestion control and the other for the combined flow control and congestion control which are based on Markov chains are designed and developed to perform the aforementioned analysis efficiently compared to time consuming detailed system simulations. In addition, the effects of the Long Term Evolution (LTE) transport network (S1and X2 interfaces) on the end user performance are investigated and analysed by introducing a novel comprehensive MAC scheduling scheme and a novel transport service differentiation model

Stochastic Dynamic Programming and Stochastic Fluid-Flow Models in the Design and Analysis of Web-Server Farms

A Web-server farm is a specialized facility designed specifically for housing Web servers catering to one or more Internet facing Web sites. In this dissertation, stochastic dynamic programming technique is used to obtain the optimal admission control policy with different classes of customers, and stochastic uid- ow models are used to compute the performance measures in the network. The two types of network traffic considered in this research are streaming (guaranteed bandwidth per connection) and elastic (shares available bandwidth equally among connections). We first obtain the optimal admission control policy using stochastic dynamic programming, in which, based on the number of requests of each type being served, a decision is made whether to allow or deny service to an incoming request. In this subproblem, we consider a xed bandwidth capacity server, which allocates the requested bandwidth to the streaming requests and divides all of the remaining bandwidth equally among all of the elastic requests. The performance metric of interest in this case will be the blocking probability of streaming traffic, which will be computed in order to be able to provide Quality of Service (QoS) guarantees. Next, we obtain bounds on the expected waiting time in the system for elastic requests that enter the system. This will be done at the server level in such a way that the total available bandwidth for the requests is constant. Trace data will be converted to an ON-OFF source and fluid- flow models will be used for this analysis. The results are compared with both the mean waiting time obtained by simulating real data, and the expected waiting time obtained using traditional queueing models. Finally, we consider the network of servers and routers within the Web farm where data from servers flows and merges before getting transmitted to the requesting users via the Internet. We compute the waiting time of the elastic requests at intermediate and edge nodes by obtaining the distribution of the out ow of the upstream node. This out ow distribution is obtained by using a methodology based on minimizing the deviations from the constituent in flows. This analysis also helps us to compute waiting times at different bandwidth capacities, and hence obtain a suitable bandwidth to promise or satisfy the QoS guarantees. This research helps in obtaining performance measures for different traffic classes at a Web-server farm so as to be able to promise or provide QoS guarantees; while at the same time helping in utilizing the resources of the server farms efficiently, thereby reducing the operational costs and increasing energy savings

Routing and Scheduling Algorithms in Resource-Limited Wireless Multi-Hop Networks

The recent advances in the area of wireless networking present novel opportunities for network operators to expand their services to infrastructure-less wireless systems.Such networks, often referred to as ad-hoc or multi-hop or peer-to-peer networks, require architectures which do not necessarily follow the cellular paradigm. They consist of entirely wireless nodes, fixed and/or mobile, that require multiple hops (and hence relaying by intermediate nodes) to transmit their messages to the desired destinations. The distinguishing features of such all-wireless network architectures give rise to new trade-offs between traditional concerns in wireless communications (such as spectral efficiency, and energy conservation) and the notions of routing, scheduling and resource allocation. The purpose of this work is to identify and study some of these novel issues, propose solutions in the context of network control and evaluate the usual network performance measures as functions of the new trade-offs.To these ends, we address first the problem of routing connection-oriented traffic with energy efficiency in all-wireless multi-hop networks. We take advantage of the flexibility of wireless nodes to transmit at different power levels and define a framework for formulating the problem of session routing from the perspective of energy expenditure. A set of heuristics are developed for determining end-to-end unicast paths with sufficient bandwidth and transceiver resources, in which nodes use local information in order to select their transmission power and bandwidth allocation. We propose a set of metrics that associate each link transmission with a cost and consider both the cases of plentiful and limited bandwidth resources, the latter jointly with a set of channel allocation algorithms. Performance is measured by call blocking probability and average consumed energy and a detailed simulation model that incorporates all the components of our algorithms has been developed and used for performance evaluation of a variety of networks.In the sequel, we propose a "blueprint" for approaching the problem of link bandwidth management in conjunction with routing, for ad-hoc wireless networks carrying packet-switched traffic. We discuss the dependencies between routing, access control and scheduling functions and propose an adaptive mechanism for solving the capacity allocation (at both the node-level and the flow-level) and the route assignment problems, that manages delays due to congestion at nodes and packet loss due to error prone wireless links, to provide improved end-to-end delay/throughput. The capacity allocations to the nodes and flows and the route assignments are iterated periodically and the adaptability of the proposed approach allows the network to respond to random channel error bursts and congestion arising from bursty and new flows

On reliable and energy efficient massive wireless communications: the road to 5G

La quinta generación de redes móviles (5G) se encuentra a la vuelta de la esquina. Se espera provea de beneficios extraordinarios a la población y que resuelva la mayoría de los problemas de las redes 4G actuales. El éxito de 5G, cuya primera fase de estandarización ha sido completada, depende de tres pilares: comunicaciones tipo-máquina masivas, banda ancha móvil mejorada y comunicaciones ultra fiables y de baja latencia (mMTC, eMBB y URLLC, respectivamente). En esta tesis nos enfocamos en el primer pilar de 5G, mMTC, pero también proveemos una solución para lograr eMBB en escenarios de distribución masiva de contenidos. Específicamente, las principales contribuciones son en las áreas de: 1) soporte eficiente de mMTC en redes celulares; 2) acceso aleatorio para el reporte de eventos en redes inalámbricas de sensores (WSNs); y 3) cooperación para la distribución masiva de contenidos en redes celulares. En el apartado de mMTC en redes celulares, esta tesis provee un análisis profundo del desempeño del procedimiento de acceso aleatorio, que es la forma mediante la cual los dispositivos móviles acceden a la red. Estos análisis fueron inicialmente llevados a cabo por simulaciones y, posteriormente, por medio de un modelo analítico. Ambos modelos fueron desarrollados específicamente para este propósito e incluyen uno de los esquemas de control de acceso más prometedores: access class barring (ACB). Nuestro modelo es uno de los más precisos que se pueden encontrar en la literatura y el único que incorpora el esquema de ACB. Los resultados obtenidos por medio de este modelo y por simulación son claros: los accesos altamente sincronizados que ocurren en aplicaciones de mMTC pueden causar congestión severa en el canal de acceso. Por otro lado, también son claros en que esta congestión se puede prevenir con una adecuada configuración del ACB. Sin embargo, los parámetros de configuración del ACB deben ser continuamente adaptados a la intensidad de accesos para poder obtener un desempeño óptimo. En la tesis se propone una solución práctica a este problema en la forma de un esquema de configuración automática para el ACB; lo llamamos ACBC. Los resultados muestran que nuestro esquema puede lograr un desempeño muy cercano al óptimo sin importar la intensidad de los accesos. Asimismo, puede ser directamente implementado en redes celulares para soportar el tráfico mMTC, ya que ha sido diseñado teniendo en cuenta los estándares del 3GPP. Además de los análisis descritos anteriormente para redes celulares, se realiza un análisis general para aplicaciones de contadores inteligentes. Es decir, estudiamos un escenario de mMTC desde la perspectiva de las WSNs. Específicamente, desarrollamos un modelo híbrido para el análisis de desempeño y la optimización de protocolos de WSNs de acceso aleatorio y basados en cluster. Los resultados muestran la utilidad de escuchar el medio inalámbrico para minimizar el número de transmisiones y también de modificar las probabilidades de transmisión después de una colisión. En lo que respecta a eMBB, nos enfocamos en un escenario de distribución masiva de contenidos, en el que un mismo contenido es enviado de forma simultánea a un gran número de usuarios móviles. Este escenario es problemático, ya que las estaciones base de la red celular no cuentan con mecanismos eficientes de multicast o broadcast. Por lo tanto, la solución que se adopta comúnmente es la de replicar e contenido para cada uno de los usuarios que lo soliciten; está claro que esto es altamente ineficiente. Para resolver este problema, proponemos el uso de esquemas de network coding y de arquitecturas cooperativas llamadas nubes móviles. En concreto, desarrollamos un protocolo para la distribución masiva de contenidos, junto con un modelo analítico para su optimización. Los resultados demuestran que el modelo propuesto es simple y preciso, y que el protocolo puede reducir el conLa cinquena generació de xarxes mòbils (5G) es troba molt a la vora. S'espera que proveïsca de beneficis extraordinaris a la població i que resolga la majoria dels problemes de les xarxes 4G actuals. L'èxit de 5G, per a la qual ja ha sigut completada la primera fase del qual d'estandardització, depén de tres pilars: comunicacions tipus-màquina massives, banda ampla mòbil millorada, i comunicacions ultra fiables i de baixa latència (mMTC, eMBB i URLLC, respectivament, per les seues sigles en anglés). En aquesta tesi ens enfoquem en el primer pilar de 5G, mMTC, però també proveïm una solució per a aconseguir eMBB en escenaris de distribució massiva de continguts. Específicament, les principals contribucions són en les àrees de: 1) suport eficient de mMTC en xarxes cel·lulars; 2) accés aleatori per al report d'esdeveniments en xarxes sense fils de sensors (WSNs); i 3) cooperació per a la distribució massiva de continguts en xarxes cel·lulars. En l'apartat de mMTC en xarxes cel·lulars, aquesta tesi realitza una anàlisi profunda de l'acompliment del procediment d'accés aleatori, que és la forma mitjançant la qual els dispositius mòbils accedeixen a la xarxa. Aquestes anàlisis van ser inicialment dutes per mitjà de simulacions i, posteriorment, per mitjà d'un model analític. Els models van ser desenvolupats específicament per a aquest propòsit i inclouen un dels esquemes de control d'accés més prometedors: el access class barring (ACB). El nostre model és un dels més precisos que es poden trobar i l'únic que incorpora l'esquema d'ACB. Els resultats obtinguts per mitjà d'aquest model i per simulació són clars: els accessos altament sincronitzats que ocorren en aplicacions de mMTC poden causar congestió severa en el canal d'accés. D'altra banda, també són clars en què aquesta congestió es pot previndre amb una adequada configuració de l'ACB. No obstant això, els paràmetres de configuració de l'ACB han de ser contínuament adaptats a la intensitat d'accessos per a poder obtindre unes prestacions òptimes. En la tesi es proposa una solució pràctica a aquest problema en la forma d'un esquema de configuració automàtica per a l'ACB; l'anomenem ACBC. Els resultats mostren que el nostre esquema pot aconseguir un acompliment molt proper a l'òptim sense importar la intensitat dels accessos. Així mateix, pot ser directament implementat en xarxes cel·lulars per a suportar el trànsit mMTC, ja que ha sigut dissenyat tenint en compte els estàndards del 3GPP. A més de les anàlisis descrites anteriorment per a xarxes cel·lulars, es realitza una anàlisi general per a aplicacions de comptadors intel·ligents. És a dir, estudiem un escenari de mMTC des de la perspectiva de les WSNs. Específicament, desenvolupem un model híbrid per a l'anàlisi de prestacions i l'optimització de protocols de WSNs d'accés aleatori i basats en clúster. Els resultats mostren la utilitat d'escoltar el mitjà sense fil per a minimitzar el nombre de transmissions i també de modificar les probabilitats de transmissió després d'una col·lisió. Pel que fa a eMBB, ens enfoquem en un escenari de distribució massiva de continguts, en el qual un mateix contingut és enviat de forma simultània a un gran nombre d'usuaris mòbils. Aquest escenari és problemàtic, ja que les estacions base de la xarxa cel·lular no compten amb mecanismes eficients de multicast o broadcast. Per tant, la solució que s'adopta comunament és la de replicar el contingut per a cadascun dels usuaris que ho sol·liciten; és clar que això és altament ineficient. Per a resoldre aquest problema, proposem l'ús d'esquemes de network coding i d'arquitectures cooperatives anomenades núvols mòbils. En concret, desenvolupem un protocol per a realitzar la distribució massiva de continguts de forma eficient, juntament amb un model analític per a la seua optimització. Els resultats demostren que el model proposat és simple i precísThe 5th generation (5G) of mobile networks is just around the corner. It is expected to bring extraordinary benefits to the population and to solve the majority of the problems of current 4th generation (4G) systems. The success of 5G, whose first phase of standardization has concluded, relies in three pillars that correspond to its main use cases: massive machine-type communication (mMTC), enhanced mobile broadband (eMBB), and ultra-reliable low latency communication (URLLC). This thesis mainly focuses on the first pillar of 5G: mMTC, but also provides a solution for the eMBB in massive content delivery scenarios. Specifically, its main contributions are in the areas of: 1) efficient support of mMTC in cellular networks; 2) random access (RA) event-reporting in wireless sensor networks (WSNs); and 3) cooperative massive content delivery in cellular networks. Regarding mMTC in cellular networks, this thesis provides a thorough performance analysis of the RA procedure (RAP), used by the mobile devices to switch from idle to connected mode. These analyses were first conducted by simulation and then by an analytical model; both of these were developed with this specific purpose and include one of the most promising access control schemes: the access class barring (ACB). To the best of our knowledge, this is one of the most accurate analytical models reported in the literature and the only one that incorporates the ACB scheme. Our results clearly show that the highly-synchronized accesses that occur in mMTC applications can lead to severe congestion. On the other hand, it is also clear that congestion can be prevented with an adequate configuration of the ACB scheme. However, the configuration parameters of the ACB scheme must be continuously adapted to the intensity of access attempts if an optimal performance is to be obtained. We developed a practical solution to this problem in the form of a scheme to automatically configure the ACB; we call it access class barring configuration (ACBC) scheme. The results show that our ACBC scheme leads to a near-optimal performance regardless of the intensity of access attempts. Furthermore, it can be directly implemented in 3rd Generation Partnership Project (3GPP) cellular systems to efficiently handle mMTC because it has been designed to comply with the 3GPP standards. In addition to the analyses described above for cellular networks, a general analysis for smart metering applications is performed. That is, we study an mMTC scenario from the perspective of event detection and reporting WSNs. Specifically, we provide a hybrid model for the performance analysis and optimization of cluster-based RA WSN protocols. Results showcase the utility of overhearing to minimize the number of packet transmissions, but also of the adaptation of transmission parameters after a collision occurs. Building on this, we are able to provide some guidelines that can drastically increase the performance of a wide range of RA protocols and systems in event reporting applications. Regarding eMBB, we focus on a massive content delivery scenario in which the exact same content is transmitted to a large number of mobile users simultaneously. Such a scenario may arise, for example, with video streaming services that offer a particularly popular content. This is a problematic scenario because cellular base stations have no efficient multicast or broadcast mechanisms. Hence, the traditional solution is to replicate the content for each requesting user, which is highly inefficient. To solve this problem, we propose the use of network coding (NC) schemes in combination with cooperative architectures named mobile clouds (MCs). Specifically, we develop a protocol for efficient massive content delivery, along with the analytical model for its optimization. Results show the proposed model is simple and accurate, and the protocol can lead to energy savings of up to 37 percent when compared to the traditional approach.Leyva Mayorga, I. (2018). On reliable and energy efficient massive wireless communications: the road to 5G [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/115484TESI

Energy Efficient and Cooperative Solutions for Next-Generation Wireless Networks

Energy efficiency is increasingly important for next-generation wireless systems due to the limited battery resources of mobile clients. While fourth generation cellular standards emphasize low client battery consumption, existing techniques do not explicitly focus on reducing power that is consumed when a client is actively communicating with the network. Based on high data rate demands of modern multimedia applications, active mode power consumption is expected to become a critical consideration for the development and deployment of future wireless technologies. Another reason for focusing more attention on energy efficient studies is given by the relatively slow progress in battery technology and the growing quality of service requirements of multimedia applications. The disproportion between demanded and available battery capacity is becoming especially significant for small-scale mobile client devices, where wireless power consumption dominates within the total device power budget. To compensate for this growing gap, aggressive improvements in all aspects of wireless system design are necessary. Recent work in this area indicates that joint link adaptation and resource allocation techniques optimizing energy efficient metrics can provide a considerable gain in client power consumption. Consequently, it is crucial to adapt state-of-the-art energy efficient approaches for practical use, as well as to illustrate the pros and cons associated with applying power-bandwidth optimization to improve client energy efficiency and develop insights for future research in this area. This constitutes the first objective of the present research. Together with energy efficiency, next-generation cellular technologies are emphasizing stronger support for heterogeneous multimedia applications. Since the integration of diverse services within a single radio platform is expected to result in higher operator profits and, at the same time, reduce network management expenses, intensive research efforts have been invested into design principles of such networks. However, as wireless resources are limited and shared by clients, service integration may become challenging. A key element in such systems is the packet scheduler, which typically helps ensure that the individual quality of service requirements of wireless clients are satisfied. In contrastingly different distributed wireless environments, random multiple access protocols are beginning to provide mechanisms for statistical quality of service assurance. However, there is currently a lack of comprehensive analytical frameworks which allow reliable control of the quality of service parameters for both cellular and local area networks. Providing such frameworks is therefore the second objective of this thesis. Additionally, the study addresses the simultaneous operation of a cellular and a local area network in spectrally intense metropolitan deployments and solves some related problems. Further improving the performance of battery-driven mobile clients, cooperative communications are sought as a promising and practical concept. In particular, they are capable of mitigating the negative effects of fading in a wireless channel and are thus expected to enhance next-generation cellular networks in terms of client spectral and energy efficiencies. At the cell edges or in areas missing any supportive relaying infrastructure, client-based cooperative techniques are becoming even more important. As such, a mobile client with poor channel quality may take advantage of neighboring clients which would relay data on its behalf. The key idea behind the concept of client relay is to provide flexible and distributed control over cooperative communications by the wireless clients themselves. By contrast to fully centralized control, this is expected to minimize overhead protocol signaling and hence ensure simpler implementation. Compared to infrastructure relay, client relay will also be cheaper to deploy. Developing the novel concept of client relay, proposing simple and feasible cooperation protocols, and analyzing the basic trade-offs behind client relay functionality become the third objective of this research. Envisioning the evolution of cellular technologies beyond their fourth generation, it appears important to study a wireless network capable of supporting machine-to-machine applications. Recent standardization documents cover a plethora of machine-to-machine use cases, as they also outline the respective technical requirements and features according to the application or network environment. As follows from this activity, a smart grid is one of the primary machine-to-machine use cases that involves meters autonomously reporting usage and alarm information to the grid infrastructure to help reduce operational cost, as well as regulate a customer's utility usage. The preliminary analysis of the reference smart grid scenario indicates weak system architecture components. For instance, the large population of machine-to-machine devices may connect nearly simultaneously to the wireless infrastructure and, consequently, suffer from excessive network entry delays. Another concern is the performance of cell-edge machine-to-machine devices with weak wireless links. Therefore, mitigating the above architecture vulnerabilities and improving the performance of future smart grid deployments is the fourth objective of this thesis. Summarizing, this thesis is generally aimed at the improvement of energy efficient properties of mobile devices in next-generation wireless networks. The related research also embraces a novel cooperation technique where clients may assist each other to increase per-client and network-wide performance. Applying the proposed solutions, the operation time of mobile clients without recharging may be increased dramatically. Our approach incorporates both analytical and simulation components to evaluate complex interactions between the studied objectives. It brings important conclusions about energy efficient and cooperative client behaviors, which is crucial for further development of wireless communications technologies