Learning for Cross-layer Resource Allocation in the Framework of Cognitive Wireless Networks

The framework of cognitive wireless networks is expected to endow wireless devices with a cognition-intelligence ability with which they can efficiently learn and respond to the dynamic wireless environment. In this dissertation, we focus on the problem of developing cognitive network control mechanisms without knowing in advance an accurate network model. We study a series of cross-layer resource allocation problems in cognitive wireless networks. Based on model-free learning, optimization and game theory, we propose a framework of self-organized, adaptive strategy learning for wireless devices to (implicitly) build the understanding of the network dynamics through trial-and-error. The work of this dissertation is divided into three parts. In the first part, we investigate a distributed, single-agent decision-making problem for real-time video streaming over a time-varying wireless channel between a single pair of transmitter and receiver. By modeling the joint source-channel resource allocation process for video streaming as a constrained Markov decision process, we propose a reinforcement learning scheme to search for the optimal transmission policy without the need to know in advance the details of network dynamics. In the second part of this work, we extend our study from the single-agent to a multi-agent decision-making scenario, and study the energy-efficient power allocation problems in a two-tier, underlay heterogeneous network and in a self-sustainable green network. For the heterogeneous network, we propose a stochastic learning algorithm based on repeated games to allow individual macro- or femto-users to find a Stackelberg equilibrium without flooding the network with local action information. For the self-sustainable green network, we propose a combinatorial auction mechanism that allows mobile stations to adaptively choose the optimal base station and sub-carrier group for transmission only from local payoff and transmission strategy information. In the third part of this work, we study a cross-layer routing problem in an interweaved Cognitive Radio Network (CRN), where an accurate network model is not available and the secondary users that are distributed within the CRN only have access to local action/utility information. In order to develop a spectrum-aware routing mechanism that is robust against potential insider attackers, we model the uncoordinated interaction between CRN nodes in the dynamic wireless environment as a stochastic game. Through decomposition of the stochastic routing game, we propose two stochastic learning algorithm based on a group of repeated stage games for the secondary users to learn the best-response strategies without the need of information flooding

Smart Grid Communications: Overview of Research Challenges, Solutions, and Standardization Activities

Optimization of energy consumption in future intelligent energy networks (or Smart Grids) will be based on grid-integrated near-real-time communications between various grid elements in generation, transmission, distribution and loads. This paper discusses some of the challenges and opportunities of communications research in the areas of smart grid and smart metering. In particular, we focus on some of the key communications challenges for realizing interoperable and future-proof smart grid/metering networks, smart grid security and privacy, and how some of the existing networking technologies can be applied to energy management. Finally, we also discuss the coordinated standardization efforts in Europe to harmonize communications standards and protocols.Comment: To be published in IEEE Communications Surveys and Tutorial

Optimization of Energy Harvesting Mobile Nodes Within Scalable Converter System Based on Reinforcement Learning

Microgrid monitoring focusing on power data, such as voltage and current, has become more significant in the development of decentralized power supply system. The power data transmission delay between distributed generator is vital for evaluating the stability and financial outcome of overall grid performance. In this thesis, both hardware and simulation has been discussed for optimizing the data packets transmission delay, energy consumption, and collision rate. To minimize the transmission delay and collision rate, state-action-reward-state-action (SARSA) and Q-learning method based on Markov decision process (MDP) model is used to search the most efficient data transmission scheme for each agent device. A training process comparison between SARSA and Q-learning is given out for representing the training speed of these two methodologies in the scenario of source-relaying-destination model. To balance the exploration and exploitation process involved in these two methods, a parameter is introduced to optimize the cost time of training process. Finally, the simulation result of average throughput and data packets collision rate in the network with 20 agent nodes is presented to indicate the application feasibility of reinforcement learning algorithm in the development of scalable network. The results show that, the average throughput and collision rate stay on the expected ideal performance level for the overall network when the number of nodes is not too large. Also, the hardware development based on Bluetooth Low Energy (BLE) is used to reveal the process of data packets transmission

Energy Efficiency

This book is one of the most comprehensive and up-to-date books written on Energy Efficiency. The readers will learn about different technologies for energy efficiency policies and programs to reduce the amount of energy. The book provides some studies and specific sets of policies and programs that are implemented in order to maximize the potential for energy efficiency improvement. It contains unique insights from scientists with academic and industrial expertise in the field of energy efficiency collected in this multi-disciplinary forum

Wide-Area Time-Synchronized Closed-Loop Control of Power Systems And Decentralized Active Distribution Networks

The rapidly expanding power system grid infrastructure and the need to reduce the occurrence of major blackouts and prevention or hardening of systems against cyber-attacks, have led to increased interest in the improved resilience of the electrical grid. Distributed and decentralized control have been widely applied to computer science research. However, for power system applications, the real-time application of decentralized and distributed control algorithms introduce several challenges. In this dissertation, new algorithms and methods for decentralized control, protection and energy management of Wide Area Monitoring, Protection and Control (WAMPAC) and the Active Distribution Network (ADN) are developed to improve the resiliency of the power system. To evaluate the findings of this dissertation, a laboratory-scale integrated Wide WAMPAC and ADN control platform was designed and implemented. The developed platform consists of phasor measurement units (PMU), intelligent electronic devices (IED) and programmable logic controllers (PLC). On top of the designed hardware control platform, a multi-agent cyber-physical interoperability viii framework was developed for real-time verification of the developed decentralized and distributed algorithms using local wireless and Internet-based cloud communication. A novel real-time multiagent system interoperability testbed was developed to enable utility independent private microgrids standardized interoperability framework and define behavioral models for expandability and plug-and-play operation. The state-of-theart power system multiagent framework is improved by providing specific attributes and a deliberative behavior modeling capability. The proposed multi-agent framework is validated in a laboratory based testbed involving developed intelligent electronic device prototypes and actual microgrid setups. Experimental results are demonstrated for both decentralized and distributed control approaches. A new adaptive real-time protection and remedial action scheme (RAS) method using agent-based distributed communication was developed for autonomous hybrid AC/DC microgrids to increase resiliency and continuous operability after fault conditions. Unlike the conventional consecutive time delay-based overcurrent protection schemes, the developed technique defines a selectivity mechanism considering the RAS of the microgrid after fault instant based on feeder characteristics and the location of the IEDs. The experimental results showed a significant improvement in terms of resiliency of microgrids through protection using agent-based distributed communication

Scalable base station switching framework for green cellular networks

With the recent unprecedented growth in the wireless market, network operators are obliged not only to find new techniques including dense deployment of base stations (BSs) in order to support high data rate services and high user density, but also to reduce the operating costs and energy consumption of various network elements. To solve these challenges, powering down certain BSs during low-traffic periods, so-called BS sleeping, has emerged as an effective green communications paradigm. While BS sleeping offers the potential to significantly lower energy consumption, it also raises many challenges, since when a BS is switched off, this can lead to, for example, coverage holes, sudden degradation in quality of service (QoS), higher transmit power dissipation in off-cell mobile stations (MSs), an inability to rapidly power up/down equipment and finally, a failure to uphold regulatory requirements. In order to realise greener network designs which both maximise energy savings whilst guaranteeing QoS, innovative BS switching mechanisms need to be developed. This thesis presents a novel BS switching framework which improves energy efficiency (EE) in comparison with existing approaches, while guaranteeing the minimum QoS and seamless services. The major technical contributions in this framework are: i) a new BS to relay station (RS) switching model where certain BSs are switched to RS mode rather than being turned off, firstly using a fixed threshold based switching algorithm utilizing temporal traffic diversity, and ii) then subsequently by means of an adaptive threshold by exploiting the inherently asymmetric traffic profile between cells, i.e., by exploiting both the temporal and spatial traffic diversity; iii) a traffic-and-interference-aware BS switching strategy that considers the impact of inter-cell interference in the decision making process to dynamically determine the best BS set to be kept active for improved EE; and finally iv) a novel scalable multimode BS switching model which enables each BS to operate in different power modes i.e., macro/micro/sleep to explore energy savings potential even at higher traffic conditions. The thesis findings conclusively confirm this new BS switching framework provides significant EE improvements from both BS and MS perspectives, under diverse network conditions and represents a notable step towards greener communications

Towards Cyber Security for Low-Carbon Transportation: Overview, Challenges and Future Directions

In recent years, low-carbon transportation has become an indispensable part as sustainable development strategies of various countries, and plays a very important responsibility in promoting low-carbon cities. However, the security of low-carbon transportation has been threatened from various ways. For example, denial of service attacks pose a great threat to the electric vehicles and vehicle-to-grid networks. To minimize these threats, several methods have been proposed to defense against them. Yet, these methods are only for certain types of scenarios or attacks. Therefore, this review addresses security aspect from holistic view, provides the overview, challenges and future directions of cyber security technologies in low-carbon transportation. Firstly, based on the concept and importance of low-carbon transportation, this review positions the low-carbon transportation services. Then, with the perspective of network architecture and communication mode, this review classifies its typical attack risks. The corresponding defense technologies and relevant security suggestions are further reviewed from perspective of data security, network management security and network application security. Finally, in view of the long term development of low-carbon transportation, future research directions have been concerned.Comment: 34 pages, 6 figures, accepted by journal Renewable and Sustainable Energy Review

Energy-Aware Adaptive Weighted Grid Clustering Algorithm for Renewable Wireless Sensor Networks

Wireless sensor networks (WSNs), built from many battery-operated sensor nodes are distributed in the environment for monitoring and data acquisition. Subsequent to the deployment of sensor nodes, the most challenging and daunting task is to enhance the energy resources for the lifetime performance of the entire WSN. In this study, we have attempted an approach based on the shortest path algorithm and grid clustering to save and renew power in a way that minimizes energy consumption and prolongs the overall network lifetime of WSNs. Initially, a wireless portable charging device (WPCD) is assumed which periodically travels on our proposed routing path among the nodes of the WSN to decrease their charge cycle time and recharge them with the help of wireless power transfer (WPT). Further, a scheduling scheme is proposed which creates clusters of WSNs. These clusters elect a cluster head among them based on the residual energy, buffer size, and distance of the head from each node of the cluster. The cluster head performs all data routing duties for all its member nodes to conserve the energy supposed to be consumed by member nodes. Furthermore, we compare our technique with the available literature by simulation, and the results showed a significant increase in the vacation time of the nodes of WSNs