Q-learning Channel Access Methods for Wireless Powered Internet of Things Networks

The Internet of Things (IoT) is becoming critical in our daily life. A key technology of interest in this thesis is Radio Frequency (RF) charging. The ability to charge devices wirelessly creates so called RF-energy harvesting IoT networks. In particular, there is a hybrid access point (HAP) that provides energy in an on-demand manner to RF-energy harvesting devices. These devices then collect data and transmit it to the HAP. In this respect, a key issue is ensuring devices have a high number of successful transmissions. There are a number of issues to consider when scheduling the transmissions of devices in the said network. First, the channel gain to/from devices varies over time. This means the efficiency to deliver energy to devices and to transmit the same amount of data is different over time. Second, during channel access, devices are not aware of the energy level of other devices nor whether they will transmit data. Third, devices have non-causal knowledge of their energy arrivals and channel gain information. Consequently, they do not know whether they should delay their transmissions in hope of better channel conditions or less contention in future time slots or doing so would result in energy overflow

環境発電を用いた多元接続通信における情報鮮度最小化ための最適送信法に関する研究

エナジーハーベスティング(EH: Energy Harvesting) 電源を具備した複数のセンサーノード(SN: Sensor Node) が，共通の1台のフュージョンセンターと通信を行う多元接続方式について，情報鮮度(AoI: Age of Information) を評価指標として議論する. 具体的には，時分割多元接続方式(TDMA: Time-Division Multiple Access)，周波数分割多元接続方式(FDMA: Frequency-Division Multiple Access)，そしてスロット化アロハの3つの方式を想定する．特に2つのグラントアクセス，TDMAとFDMAの場合には，AoIの凸性を利用しAoIを最小にするための凸最適化問題を定義する．定義する最適化問題に対し，TDMAでは観測情報を送信する準備が整ったSN から順番に送信する貪欲アルゴリズムを適用し，FDMAでは，交互方向乗数法，および二分探索法を用いて，最小AoIを導出する．一方，SNの送信すべきデータ量が少ない場合には，無線資源割り当てに必要なオーバヘッドが無視できず，排他的な割り当てを行う必要のないグラントフリーアクセスがグラントアクセスと比較して有利となる場合がある．そこで，グラントフリーアクセスの1つであるスロット化アロハのAoIをTDMA，FDMAとは異なる形で定式化するとともに，EHを考慮した場合のデータ伝送処理についても言及する．数値結果では，TDMAとFDMAそれぞれの場合について，各SNに送信時間，周波数帯域割り当てを行うための最適化問題を解き，2つのグラントアクセスのAoIを比較する．さらに，スロット化アロハが達成可能なAoIを示し，オーバヘッドを考慮したTDMAと比較することで，オーバヘッドと送信時間スロットの割合に対し，ランダムアクセスがより小さいAoIを達成する領域を明らかにする．電気通信大学202

Practical Evaluation of Low-complexity Medium Access Control Protocols for Wireless Sensor Networks

This thesis studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks (WSNs), whereby Reinforcement Learning (RL) is employed as an intelligent transmission strategy. RL is applied to framed slotted-ALOHA to provide perfect scheduling. The system converges to a steady state of a unique transmission slot assigned per node in single-hop and multi-hop communication if there is sufficient number of slots available in the network, thereby achieving the optimum performance. The stability of the system against possible changes in the environment and changing channel conditions is studied. A Markov model is provided to represent the learning behaviour, which is also used to predict how the system loses its operation after convergence. Novel schemes are proposed to protect the lifetime of the system when the environment and channel conditions are insufficient to maintain the operation of the system. Taking real sensor platform architectures into consideration, the practicality of MAC protocols for WSNs must be considered based on hardware limitations/constraints. Therefore, the performance of the schemes developed is demonstrated through extensive simulations and evaluations in various test-beds. Practical evaluations show that RL-based schemes provide a high level of flexibility for hardware implementation

Performance studies of wireless multihop networks

Wireless multihop networks represent a fundamental step in the evolution of wireless communications, a step that has proven challenging. Such networks give rise to a wide range of novel performance and design problems, most of which are of a geometric nature. This dissertation addresses a selection of such problems. The first part of this thesis presents studies in which the network nodes are assumed to receive signals sufficiently clearly only from within some fixed range of operation. Using this simple model, the first two problems addressed are to predict the probabilities that a network with randomly placed nodes is connected or completely covers a given target domain, respectively. These problems are equivalent to determining the probability distribution of the minimal range providing connectivity or coverage. Algorithms for determining these threshold ranges for a given set of network nodes are developed. Because of the complex nature of these problems in finite settings, they are both approached by empirically modeling the convergence of these distributions to their known asymptotic limits. Next, a novel optimization problem is presented, in which the task is to make a given disconnected network into a connected one by adding a minimal number of additional nodes to the network, and heuristic algorithms are proposed for this problem. In the second part, these networks are studied in the context of a more realistic model in which the condition for successful communication between network nodes is expressed as an explicit minimum value for the received signal-to-noise-and-interference ratio. The notion of the threshold range for connectivity is first generalized to this network model. Because connectivity is now affected by medium access control (MAC), two alternative MAC schemes are considered. Finally, an infinite random network employing slotted Aloha is studied under this model. Since the probability of successful reception in a random time slot is a function of the locations of other nodes, this temporal probability is a random variable with its own probability distribution over different node configurations. Numerical approximations for evaluating both the mean and the tail probability of this distribution are developed. The accuracy of these approximations can be improved indefinitely, at the cost of numerical computations.reviewe

Energy Harvesting Wireless Communications: A Review of Recent Advances

This article summarizes recent contributions in the broad area of energy harvesting wireless communications. In particular, we provide the current state of the art for wireless networks composed of energy harvesting nodes, starting from the information-theoretic performance limits to transmission scheduling policies and resource allocation, medium access and networking issues. The emerging related area of energy transfer for self-sustaining energy harvesting wireless networks is considered in detail covering both energy cooperation aspects and simultaneous energy and information transfer. Various potential models with energy harvesting nodes at different network scales are reviewed as well as models for energy consumption at the nodes.Comment: To appear in the IEEE Journal of Selected Areas in Communications (Special Issue: Wireless Communications Powered by Energy Harvesting and Wireless Energy Transfer