Cognitive Radio Networking with Cooperative Relaying and Energy Harvesting

Cognitive radio networks with energy harvesting result in efficient use of both energy and spectrum. By using cooperative relaying, another feature can be achieved, which is the high diversity gain. In this paper, an energy harvesting underlay cognitive radio relaying network is investigated. In this underlay cognitive radio scheme, secondary users are allowed to access the spectrum, respecting a certain primary interference threshold. The secondary nodes employ decode-and-forward relaying in order to maximize the total received data by optimizing their transmit powers. In this context, both the secondary source and relay harvest energy from renewable sources and store it in finite batteries. They are also capable of buffering data in infinite capacity buffers. We derive closed form expressions for transmit power of secondary source and relay that maximize the secondary network throughput. Projected subgradient method is used to find the power allocated to the secondary network. Numerical simulations are conducted to study the performance of the proposed system. Comparisons are made between the proposed system and other conventional scenarios, and it is observed that when the required signal-to interference-plus-noise ratio (SINR) at the primary receiver is high, the proposed harvesting- based scheme and conventional-based scheme perform similarly

An Actor-Critic Reinforcement Learning Approach for Energy Harvesting Communications Systems

Energy harvesting communications systems are able to provide high quality communications services using green energy sources. This paper presents an autonomous energy harvesting communications system that is able to adapt to any environment, and optimize its behavior with experience to maximize the valuable received data. The considered system is a point-to-point energy harvesting communications system consisting of a source and a destination, and working in an unknown and uncertain environment. The source is an energy harvesting node capable of harvesting solar energy and storing it in a finite capacity battery. Energy can be harvested, stored, and used from continuous ranges of energy values. Channel gains can take any value within a continuous range. Since exact information about future channel gains and harvested energy is unavailable, an architecture based on actor-critic reinforcement learning is proposed to learn a close-to-optimal transmission power allocation policy. The actor uses a stochastic parameterized policy to select actions at states stochastically. The policy is modeled by a normal distribution with a parameterized mean and standard deviation. The actor uses policy gradient to optimize the policy’s parameters. The critic uses a three layer neural network to approximate the action-value function, and to evaluate the optimized policy. Simulation results evaluate the proposed architecture for actor-critic learning, and shows its ability to improve its performance with experience

Enhancing the performance of energy harvesting wireless communications using optimization and machine learning

The motivation behind this thesis is to provide efficient solutions for energy harvesting communications. Firstly, an energy harvesting underlay cognitive radio relaying network is investigated. In this context, the secondary network is an energy harvesting network. Closed-form expressions are derived for transmission power of secondary source and relay that maximizes the secondary network throughput. Secondly, a practical scenario in terms of information availability about the environment is investigated. We consider a communications system with a source capable of harvesting solar energy. Two cases are considered based on the knowledge availability about the underlying processes. When this knowledge is available, an algorithm using this knowledge is designed to maximize the expected throughput, while reducing the complexity of traditional methods. For the second case, when the knowledge about the underlying processes is unavailable, reinforcement learning is used. Thirdly, a number of learning architectures for reinforcement learning are introduced. They are called selector-actor-critic, tuner-actor-critic, and estimator-selector-actor-critic. The goal of the selector-actor-critic architecture is to increase the speed and the efficiency of learning an optimal policy by approximating the most promising action at the current state. The tuner-actor-critic aims at improving the learning process by providing the actor with a more accurate estimation about the value function. Estimator-selector-actor-critic is introduced to support intelligent agents. This architecture mimics rational humans in the way of analyzing available information, and making decisions. Then, a harvesting communications system working in an unknown environment is evaluated when it is supported by the proposed architectures. Fourthly, a realistic energy harvesting communications system is investigated. The state and action spaces of the underlying Markov decision process are continuous. Actor-critic is used to optimize the system performance. The critic uses a neural network to approximate the action-value function. The actor uses policy gradient to optimize the policy\u27s parameters to maximize the throughput