Machine Learning-Enabled Resource Allocation for Underlay Cognitive Radio Networks

Due to the rapid growth of new wireless communication services and applications, much attention has been directed to frequency spectrum resources and the way they are regulated. Considering that the radio spectrum is a natural limited resource, supporting the ever increasing demands for higher capacity and higher data rates for diverse sets of users, services and applications is a challenging task which requires innovative technologies capable of providing new ways of efficiently exploiting the available radio spectrum. Consequently, dynamic spectrum access (DSA) has been proposed as a replacement for static spectrum allocation policies. The DSA is implemented in three modes including interweave, overlay and underlay mode [1]. The key enabling technology for DSA is cognitive radio (CR), which is among the core prominent technologies for the next generation of wireless communication systems. Unlike conventional radio which is restricted to only operate in designated spectrum bands, a CR has the capability to operate in different spectrum bands owing to its ability in sensing, understanding its wireless environment, learning from past experiences and proactively changing the transmission parameters as needed. These features for CR are provided by an intelligent software package called the cognitive engine (CE). In general, the CE manages radio resources to accomplish cognitive functionalities and allocates and adapts the radio resources to optimize the performance of the network. Cognitive functionality of the CE can be achieved by leveraging machine learning techniques. Therefore, this thesis explores the application of two machine learning techniques in enabling the cognition capability of CE. The two considered machine learning techniques are neural network-based supervised learning and reinforcement learning. Specifically, this thesis develops resource allocation algorithms that leverage the use of machine learning techniques to find the solution to the resource allocation problem for heterogeneous underlay cognitive radio networks (CRNs). The proposed algorithms are evaluated under extensive simulation runs. The first resource allocation algorithm uses a neural network-based learning paradigm to present a fully autonomous and distributed underlay DSA scheme where each CR operates based on predicting its transmission effect on a primary network (PN). The scheme is based on a CE with an artificial neural network that predicts the adaptive modulation and coding configuration for the primary link nearest to a transmitting CR, without exchanging information between primary and secondary networks. By managing the effect of the secondary network (SN) on the primary network, the presented technique maintains the relative average throughput change in the primary network within a prescribed maximum value, while also finding transmit settings for the CRs that result in throughput as large as allowed by the primary network interference limit. The second resource allocation algorithm uses reinforcement learning and aims at distributively maximizing the average quality of experience (QoE) across transmission of CRs with different types of traffic while satisfying a primary network interference constraint. To best satisfy the QoE requirements of the delay-sensitive type of traffics, a cross-layer resource allocation algorithm is derived and its performance is compared against a physical-layer algorithm in terms of meeting end-to-end traffic delay constraints. Moreover, to accelerate the learning performance of the presented algorithms, the idea of transfer learning is integrated. The philosophy behind transfer learning is to allow well-established and expert cognitive agents (i.e. base stations or mobile stations in the context of wireless communications) to teach newly activated and naive agents. Exchange of learned information is used to improve the learning performance of a distributed CR network. This thesis further identifies the best practices to transfer knowledge between CRs so as to reduce the communication overhead. The investigations in this thesis propose a novel technique which is able to accurately predict the modulation scheme and channel coding rate used in a primary link without the need to exchange information between the two networks (e.g. access to feedback channels), while succeeding in the main goal of determining the transmit power of the CRs such that the interference they create remains below the maximum threshold that the primary network can sustain with minimal effect on the average throughput. The investigations in this thesis also provide a physical-layer as well as a cross-layer machine learning-based algorithms to address the challenge of resource allocation in underlay cognitive radio networks, resulting in better learning performance and reduced communication overhead

Planning and dynamic spectrum management in heterogeneous mobile networks with QoE optimization

The radio and network planning and optimisation are continuous processes that do not end after the network has been launched. To achieve the best trade-offs, especially between quality and costs, operators make use of several coverage and capacity enhancement methods. The research from this thesis proposes methods such as the implementation of cell zooming and Relay Stations (RSs) with dynamic sleep modes and Carrier Aggregation (CA) for coverage and capacity enhancements. Initially, a survey is presented on ubiquitous mesh networks implementation scenarios and an updated characterization of requirements for services and applications is proposed. The performance targets for the key parameters, delay, delay variation, information loss and throughput have been addressed for all types of services. Furthermore, with the increased competition, mobile operator’s success does not only depend on how good the offered Quality of Service (QoS) is, but also if it meets the end user’s expectations, i.e., Quality of Experience (QoE). In this context, a model for the mapping between QoS parameters and QoE has been proposed for multimedia traffic. The planning and optimization of fixed Worldwide Interoperability for Microwave Access (WiMAX) networks with RSs in conjunction with cell zooming has been addressed. The challenging case of a propagation measurement-based scenario in the hilly region of Covilhã has been considered. A cost/revenue function has been developed by taking into account the cost of building and maintaining the infrastructure with the use of RSs. This part of the work also investigates the energy efficiency and economic implications of the use of power saving modes for RSs in conjunction with cell zooming. Assuming that the RSs can be switched-off or zoomed out to zero in periods when the traffic exchange is low, such as nights and weekends, it has been shown that energy consumption may be reduced whereas cellular coverage and capacity, as well as economic performance may be improved. An integrated Common Radio Resource Management (iCRRM) entity is proposed that implements inter-band CA by performing scheduling between two Long Term Evolution – Advanced (LTE-A) Component Carriers (CCs). Considering the bandwidths available in Portugal, the 800 MHz and 2.6 GHz CCs have been considered whilst mobile video traffic is addressed. Through extensive simulations it has been found that the proposed multi-band schedulers overcome the capacity of LTE systems without CA. Result shown a clear improvement of the QoS, QoE and economic trade-off with CA

LTE Optimization and Resource Management in Wireless Heterogeneous Networks

Mobile communication technology is evolving with a great pace. The development of the Long Term Evolution (LTE) mobile system by 3GPP is one of the milestones in this direction. This work highlights a few areas in the LTE radio access network where the proposed innovative mechanisms can substantially improve overall LTE system performance. In order to further extend the capacity of LTE networks, an integration with the non-3GPP networks (e.g., WLAN, WiMAX etc.) is also proposed in this work. Moreover, it is discussed how bandwidth resources should be managed in such heterogeneous networks. The work has purposed a comprehensive system architecture as an overlay of the 3GPP defined SAE architecture, effective resource management mechanisms as well as a Linear Programming based analytical solution for the optimal network resource allocation problem. In addition, alternative computationally efficient heuristic based algorithms have also been designed to achieve near-optimal performance

Cross-Layer QoE Improvement with Dynamic Spectrum Allocation in OFDM-Based Cognitive Radio.

PhDRapid development of devices and applications results in dramatic growth of wireless tra c, which leads to increasing demand on wire- less spectrum resources. Current spectrum resource allocation pol- icy causes low e ciency in licensed spectrum bands. Cognitive Ra- dio techniques are a promising solution to the problem of spectrum scarcity and low spectrum utilisation. Especially, OFDM based Cog- nitive Radio has received much research interest due to its exibility in enabling dynamic resource allocation. Extensive research has shown how to optimise Cognitive Radio networks in many ways, but there has been little consideration of the real-time packet level performance of the network. In such a situation, the Quality of Service metrics of the Secondary Network are di cult to guarantee due to uctuating resource availability; nevertheless QoS metric evaluation is actually a very important factor for the success of Cognitive Radio. Quality of Experience is also gaining interest due to its focus on the users' per- ceived quality, and this opens up a new perspective on evaluating and improving wireless networks performance. The main contributions of this thesis include: it focuses on the real-time packet level QoS (packet delay and loss) performance of Cognitive Radio networks, and eval- uates the e ects on QoS of several typical non-con gurable factors including secondary user service types, primary user activity patterns and user distance from base station. Furthermore, the evaluation results are uni ed and represented using QoE through existing map- ping techniques. Based on the QoE evaluation, a novel cross layer RA scheme is proposed to dynamically compensate user experience, and this is shown to signi cantly improve QoE in scenarios where traditional RA schemes fail to provide good user experience

Game theory for collaboration in future networks

Cooperative strategies have the great potential of improving network performance and spectrum utilization in future networking environments. This new paradigm in terms of network management, however, requires a novel design and analysis framework targeting a highly flexible networking solution with a distributed architecture. Game Theory is very suitable for this task, since it is a comprehensive mathematical tool for modeling the highly complex interactions among distributed and intelligent decision makers. In this way, the more convenient management policies for the diverse players (e.g. content providers, cloud providers, home providers, brokers, network providers or users) should be found to optimize the performance of the overall network infrastructure. The authors discuss in this chapter several Game Theory models/concepts that are highly relevant for enabling collaboration among the diverse players, using different ways to incentivize it, namely through pricing or reputation. In addition, the authors highlight several related open problems, such as the lack of proper models for dynamic and incomplete information games in this area.info:eu-repo/semantics/acceptedVersio

Game theory for cooperation in multi-access edge computing

Cooperative strategies amongst network players can improve network performance and spectrum utilization in future networking environments. Game Theory is very suitable for these emerging scenarios, since it models high-complex interactions among distributed decision makers. It also finds the more convenient management policies for the diverse players (e.g., content providers, cloud providers, edge providers, brokers, network providers, or users). These management policies optimize the performance of the overall network infrastructure with a fair utilization of their resources. This chapter discusses relevant theoretical models that enable cooperation amongst the players in distinct ways through, namely, pricing or reputation. In addition, the authors highlight open problems, such as the lack of proper models for dynamic and incomplete information scenarios. These upcoming scenarios are associated to computing and storage at the network edge, as well as, the deployment of large-scale IoT systems. The chapter finalizes by discussing a business model for future networks.info:eu-repo/semantics/acceptedVersio

Anticipatory Buffer Control and Quality Selection for Wireless Video Streaming

Video streaming is in high demand by mobile users, as recent studies indicate. In cellular networks, however, the unreliable wireless channel leads to two major problems. Poor channel states degrade video quality and interrupt the playback when a user cannot sufficiently fill its local playout buffer: buffer underruns occur. In contrast to that, good channel conditions cause common greedy buffering schemes to pile up very long buffers. Such over-buffering wastes expensive wireless channel capacity. To keep buffering in balance, we employ a novel approach. Assuming that we can predict data rates, we plan the quality and download time of the video segments ahead. This anticipatory scheduling avoids buffer underruns by downloading a large number of segments before a channel outage occurs, without wasting wireless capacity by excessive buffering. We formalize this approach as an optimization problem and derive practical heuristics for segmented video streaming protocols (e.g., HLS or MPEG DASH). Simulation results and testbed measurements show that our solution essentially eliminates playback interruptions without significantly decreasing video quality

QoE-driven LTE Downlink Scheduling for Multimedia Services

The significant growth in multimedia services and traffic (e.g. VoIP, video streaming and video gaming) in current and emerging mobile networks including the latest 4G Long-Term Evolution (LTE) networks and the rising user expectation for high Quality of Experience (QoE) for these services have posed real challenges to network operators and service providers. One of the key challenges is how to bring multimedia services to the end-user over resource-constrained mobile networks with a satisfactory QoE. Cost-effective solutions are needed for network operators to improve the bandwidth usage of these mobile networks. Therefore, scheduling schemes are of extreme importance in LTE, where scheduling algorithms are responsible for the overall efficiency of resource allocation in an LTE system. The aim of the project is to develop novel QoE-driven scheduling algorithms for improving system capacity in delivering multimedia services over downlink 3GPP LTE. This is to move away from traditional QoS-driven scheduling schemes to a QoE-driven scheme which guarantee end-user satisfaction in resource allocation. The main contributions of the thesis are threefold: 1. Performance of several existing scheduling algorithms for VoIP applications was evaluated thoroughly in terms of QoE metric (i.e. MOS), instead of QoS metrics (e.g. packet loss and delay). Using QoE metrics instead of QoS ones will facilitate the development of QoE-driven scheduling schemes in order to achieve optimised end-user experiences or optimised mobile system capacity. 2. A novel QoE-driven LTE downlink scheduling scheme for VoIP application was developed to maximize the number of users per cell at an acceptable MOS score. The proposed scheme achieved significant improvement in cell capacity at an acceptable quality (75% compared to MLWDF, and 250% compared to PF and EXP-PF in all three lower speed scenarios considered). 3. A QoE-driven LTE downlink scheduling scheme for multiservice multimedia applications was developed to improve the cell capacity with satisfactory QoE for both VoIP and video streaming services. The proposed algorithm performed well in a pedestrian scenario increasing cell capacity to double for video stream with ‘Rapid Movement’ (RM) content. For ‘Medium Movement’ (MM) video content, the capacity was increased about 20% compared to MLWDF and by 40% compared to EXP-PF. In a vehicular scenario, the proposed scheme managed to enhance the cell capacity for MM video stream case. The project has led to three publications (IEEE Globecom’12 – QoEMC Workshop, IEEE CCNC’15 and IEEE MMTC E-letter/May-2015). A journal paper is in preparation.The Public Authority for Applied Education and Training , Kuwai

Optimisation des Systèmes Partiellement Observables dans les Réseaux Sans-fil (Théorie des jeux, Auto-adaptation et Apprentissage)

La dernière décennie a vu l'émergence d'Internet et l'apparition des applications multimédia qui requièrent de plus en plus de bande passante, ainsi que des utilisateurs qui exigent une meilleure qualité de service. Dans cette perspective, beaucoup de travaux ont été effectués pour améliorer l'utilisation du spectre sans fil.Le sujet de ma thèse de doctorat porte sur l'application de la théorie des jeux, la théorie des files d'attente et l'apprentissage dans les réseaux sans fil,en particulier dans des environnements partiellement observables. Nous considérons différentes couches du modèle OSI. En effet, nous étudions l'accès opportuniste au spectre sans fil à la couche MAC en utilisant la technologie des radios cognitifs (CR). Par la suite, nous nous concentrons sur le contrôle de congestion à la couche transport, et nous développons des mécanismes de contrôle de congestion pour le protocole TCP.Since delay-sensitive and bandwidth-intense multimedia applications have emerged in the Internet, the demand for network resources has seen a steady increase during the last decade. Specifically, wireless networks have become pervasive and highly populated.These motivations are behind the problems considered in this dissertation.The topic of my PhD is about the application of game theory, queueing theory and learning techniques in wireless networks under some QoS constraints, especially in partially observable environments.We consider different layers of the protocol stack. In fact, we study the Opportunistic Spectrum Access (OSA) at the Medium Access Control (MAC) layer through Cognitive Radio (CR) approaches.Thereafter, we focus on the congestion control at the transport layer, and we develop some congestion control mechanisms under the TCP protocol.The roadmap of the research is as follows. Firstly, we focus on the MAC layer, and we seek for optimal OSA strategies in CR networks. We consider that Secondary Users (SUs) take advantage of opportunities in licensed channels while ensuring a minimum level of QoS. In fact, SUs have the possibility to sense and access licensed channels, or to transmit their packets using a dedicated access (like 3G). Therefore, a SU has two conflicting goals: seeking for opportunities in licensed channels, but spending energy for sensing those channels, or transmitting over the dedicated channel without sensing, but with higher transmission delay. We model the slotted and the non-slotted systems using a queueing framework. Thereafter, we analyze the non-cooperative behavior of SUs, and we prove the existence of a Nash equilibrium (NE) strategy. Moreover, we measure the gap of performance between the centralized and the decentralized systems using the Price of Anarchy (PoA).Even if the OSA at the MAC layer was deeply investigated in the last decade, the performance of SUs, such as energy consumption or Quality of Service (QoS) guarantee, was somehow ignored. Therefore, we study the OSA taking into account energy consumption and delay. We consider, first, one SU that access opportunistically licensed channels, or transmit its packets through a dedicated channel. Due to the partial spectrum sensing, the state of the spectrum is partially observable. Therefore, we use the Partially Observable Markov Decision Process (POMDP) framework to design an optimal OSA policy for SUs. Specifically, we derive some structural properties of the value function, and we prove that the optimal OSA policy has a threshold structure.Thereafter, we extend the model to the context of multiple SUs. We study the non-cooperative behavior of SUs and we prove the existence of a NE. Moreover, we highlight a paradox in this situation: more opportunities in the licensed spectrum may lead to worst performances for SUs. Thereafter, we focus on the study of spectrum management issues. In fact, we introduce a spectrum manager to the model, and we analyze the hierarchical game between the network manager and SUs.Finally, we focus on the transport layer and we study the congestion control for wireless networks under some QoS and Quality of Experience (QoE) constraints. Firstly, we propose a congestion control algorithm that takes into account applications' parameters and multimedia quality. In fact, we consider that network users maximize their expected multimedia quality by choosing the congestion control strategy. Since users ignore the congestion status at bottleneck links, we use a POMDP framework to determine the optimal congestion control strategy.Thereafter, we consider a subjective measure of the multimedia quality, and we propose a QoE-based congestion control algorithm. This algorithm bases on QoE feedbacks from receivers in order to adapt the congestion window size. Note that the proposed algorithms are designed based on some learning methods in order to face the complexity of solving POMDP problems.AVIGNON-Bib. numérique (840079901) / SudocSudocFranceF