An improved multi-agent simulation methodology for modelling and evaluating wireless communication systems resource allocation algorithms

Multi-Agent Systems (MAS) constitute a well known approach in modelling dynamical real world systems. Recently, this technology has been applied to Wireless Communication Systems (WCS), where efficient resource allocation is a primary goal, for modelling the physical entities involved, like Base Stations (BS), service providers and network operators. This paper presents a novel approach in applying MAS methodology to WCS resource allocation by modelling more abstract entities involved in WCS operation, and especially the concurrent network procedures (services). Due to the concurrent nature of a WCS, MAS technology presents a suitable modelling solution. Services such as new call admission, handoff, user movement and call termination are independent to one another and may occur at the same time for many different users in the network. Thus, the required network procedures for supporting the above services act autonomously, interact with the network environment (gather information such as interference conditions), take decisions (e.g. call establishment), etc, and can be modelled as agents. Based on this novel simulation approach, the agent cooperation in terms of negotiation and agreement becomes a critical issue. To this end, two negotiation strategies are presented and evaluated in this research effort and among them the distributed negotiation and communication scheme between network agents is presented to be highly efficient in terms of network performance. The multi-agent concept adapted to the concurrent nature of large scale WCS is, also, discussed in this paper

Energy-Efficient Resource Allocation for Device-to-Device Underlay Communication

Device-to-device (D2D) communication underlaying cellular networks is expected to bring significant benefits for utilizing resources, improving user throughput and extending battery life of user equipments. However, the allocation of radio and power resources to D2D communication needs elaborate coordination, as D2D communication can cause interference to cellular communication. In this paper, we study joint channel and power allocation to improve the energy efficiency of user equipments. To solve the problem efficiently, we introduce an iterative combinatorial auction algorithm, where the D2D users are considered as bidders that compete for channel resources, and the cellular network is treated as the auctioneer. We also analyze important properties of D2D underlay communication, and present numerical simulations to verify the proposed algorithm.Comment: IEEE Transactions on Wireless Communication

Resource Management in Delay Tolerant Networks and Smart Grid

In recent years, significant advances have been achieved in communication networks and electric power systems. Communication networks are developed to provide services within not only well-connected network environments such as wireless local area networks, but also challenged network environments where continuous end-to-end connections can hardly be established between information sources and destinations. Delay tolerant network (DTN) is proposed to achieve this objective by utilizing a store-carry-and-forward routing scheme. However, as the network connections in DTNs are intermittent in nature, the management of network resources such as communication bandwidth and buffer storage becomes a challenging issue. On the other hand, the smart grid is to explore information and communication technologies in electric power grids to achieve electricity delivery in a more efficient and reliable way. A high penetration level of electric vehicles and renewable power generation is expected in the future smart grid. However, the randomness of electric vehicle mobility and the intermittency of renewable power generation bring new challenges to the resources management in the smart grid, such as electric power, energy storage, and communication bandwidth management. This thesis consists of two parts. In part I, we focus on the resource management in DTNs. Specifically, we investigate data dissemination and on-demand data delivery which are two of the major data services in DTNs. Two kinds of mobile nodes are considered for the two types of services which correspond to the pedestrians and high-speed train passengers, respectively. For pedestrian nodes, the roadside wireless local area networks are used as an auxiliary communication infrastructure for data service delivery. We consider a cooperative data dissemination approach with a packet pre-downloading mechanism and propose a double-loop receiver-initiated medium access control scheme to resolve the channel contention among multiple direct/relay links and exploit the predictable traffic characteristics as a result of packet pre-downloading. For high-speed train nodes, we investigate on-demand data service delivery via a cellular/infostation integrated network. The optimal resource allocation problem is formulated by taking account of the intermittent network connectivity and multi-service demands. In order to achieve efficient resource allocation with low computational complexity, the original problem is transformed into a single-machine preemptive scheduling problem and an online resource allocation algorithm is proposed. If the link from the backbone network to an infostation is a bottleneck, a service pre-downloading algorithm is also proposed to facilitate the resource allocation. In part II, we focus on resource management in the smart grid. We first investigate the optimal energy delivery for plug-in hybrid electric vehicles via vehicle-to-grid systems. A dynamic programming formulation is established by considering the bidirectional energy flow, non-stationary energy demand, battery characteristics, and time-of-use electricity price. We prove the optimality of a state-dependent double-threshold policy based on the stochastic inventory theory. A modified backward iteration algorithm is devised for practical applications, where an exponentially weighted moving average algorithm is used to estimate the statistics of vehicle mobility and energy demand. Then, we propose a decentralized economic dispatch approach for microgrids such that the optimal decision on power generation is made by each distributed generation unit locally via multiagent coordination. To avoid a slow convergence speed of multiagent coordination, we propose a heterogeneous wireless network architecture for microgrids. Two multiagent coordination schemes are proposed for the single-stage and hierarchical operation modes, respectively. The optimal number of activated cellular communication devices is obtained based on the tradeoff between communication and generation costs

Learning and Reasoning Strategies for User Association in Ultra-dense Small Cell Vehicular Networks

Recent vehicular ad hoc networks research has been focusing on providing intelligent transportation services by employing information and communication technologies on road transport. It has been understood that advanced demands such as reliable connectivity, high user throughput, and ultra-low latency required by these services cannot be met using traditional communication technologies. Consequently, this thesis reports on the application of artificial intelligence to user association as a technology enabler in ultra-dense small cell vehicular networks. In particular, the work focuses on mitigating mobility-related concerns and networking issues at different mobility levels by employing diverse heuristic as well as reinforcement learning (RL) methods. Firstly, driven by rapid fluctuations in the network topology and the radio environment, a conventional, three-step sequence user association policy is designed to highlight and explore the impact of vehicle speed and different performance indicators on network quality of service (QoS) and user experience. Secondly, inspired by control-theoretic models and dynamic programming, a real-time controlled feedback user association approach is proposed. The algorithm adapts to the changing vehicular environment by employing derived network performance information as a heuristic, resulting in improved network performance. Thirdly, a sequence of novel RL based user association algorithms are developed that employ variable learning rate, variable rewards function and adaptation of the control feedback framework to improve the initial and steady-state learning performance. Furthermore, to accelerate the learning process and enhance the adaptability and robustness of the developed RL algorithms, heuristically accelerated RL and case-based transfer learning methods are employed. A comprehensive, two-tier, event-based, system level simulator which is an integration of a dynamic vehicular network, a highway, and an ultra-dense small cell network is developed. The model has enabled the analysis of user mobility effects on the network performance across different mobility levels as well as served as a firm foundation for the evaluation of the empirical properties of the investigated approaches

Distributed Artificial Intelligence Solution for D2D Communication in 5G Networks

Device to Device (D2D) Communication is one of the technology components of the evolving 5G architecture, as it promises improvements in energy efficiency, spectral efficiency, overall system capacity, and higher data rates. The above noted improvements in network performance spearheaded a vast amount of research in D2D, which have identified significant challenges that need to be addressed before realizing their full potential in emerging 5G Networks. Towards this end, this paper proposes the use of a distributed intelligent approach to control the generation of D2D networks. More precisely, the proposed approach uses Belief-Desire-Intention (BDI) intelligent agents with extended capabilities (BDIx) to manage each D2D node independently and autonomously, without the help of the Base Station. The paper includes detailed algorithmic description for the decision of transmission mode, which maximizes the data rate, minimizes the power consumptions, while taking into consideration the computational load. Simulations show the applicability of BDI agents in jointly solving D2D challenges.Comment: 10 pages,9 figure

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig