Use of Q-Learning Approaches for Practical Medium Access Control in Wireless Sensor Networks

This paper studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks. Q-Learning is employed as an intelligent transmission strategy. We review the existing MAC protocols in the context of Q-learning. A recently-proposed, ALOHA and Q-Learning based MAC scheme, ALOHA-Q, is considered which improves the channel performance significantly with a key benefit of simplicity. Practical implementation issues of ALOHA-Q are studied. We demonstrate the performance of the ALOHA-Q through extensive simulations and evaluations in various testbeds. A new exploration/exploitation method is proposed to strengthen the merits of the ALOHA-Q against dynamic the channel and environment conditions. © 2016 Elsevier Lt

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

Practical Evaluation of Low-complexity Medium Access Control Protocols for Wireless Sensor Networks

This thesis studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks (WSNs), whereby Reinforcement Learning (RL) is employed as an intelligent transmission strategy. RL is applied to framed slotted-ALOHA to provide perfect scheduling. The system converges to a steady state of a unique transmission slot assigned per node in single-hop and multi-hop communication if there is sufficient number of slots available in the network, thereby achieving the optimum performance. The stability of the system against possible changes in the environment and changing channel conditions is studied. A Markov model is provided to represent the learning behaviour, which is also used to predict how the system loses its operation after convergence. Novel schemes are proposed to protect the lifetime of the system when the environment and channel conditions are insufficient to maintain the operation of the system. Taking real sensor platform architectures into consideration, the practicality of MAC protocols for WSNs must be considered based on hardware limitations/constraints. Therefore, the performance of the schemes developed is demonstrated through extensive simulations and evaluations in various test-beds. Practical evaluations show that RL-based schemes provide a high level of flexibility for hardware implementation

Intelligent Medium Access Control Protocols for Wireless Sensor Networks

The main contribution of this thesis is to present the design and evaluation of intelligent MAC protocols for Wireless Sensor Networks (WSNs). The objective of this research is to improve the channel utilisation of WSNs while providing flexibility and simplicity in channel access. As WSNs become an efficient tool for recognising and collecting various types of information from the physical world, sensor nodes are expected to be deployed in diverse geographical environments including volcanoes, jungles, and even rivers. Consequently, the requirements for the flexibility of deployment, the simplicity of maintenance, and system self-organisation are put into a higher level. A recently developed reinforcement learning-based MAC scheme referred as ALOHA-Q is adopted as the baseline MAC scheme in this thesis due to its intelligent collision avoidance feature, on-demand transmission strategy and relatively simple operation mechanism. Previous studies have shown that the reinforcement learning technique can considerably improve the system throughput and significantly reduce the probability of packet collisions. However, the implementation of reinforcement learning is based on assumptions about a number of critical network parameters. That impedes the usability of ALOHA-Q. To overcome the challenges in realistic scenarios, this thesis proposes numerous novel schemes and techniques. Two types of frame size evaluation schemes are designed to deal with the uncertainty of node population in single-hop systems, and the unpredictability of radio interference and node distribution in multi-hop systems. A slot swapping techniques is developed to solve the hidden node issue of multi-hop networks. Moreover, an intelligent frame adaptation scheme is introduced to assist sensor nodes to achieve collision-free scheduling in cross chain networks. The combination of these individual contributions forms state of the art MAC protocols, which offers a simple, intelligent and distributed solution to improving the channel utilisation and extend the lifetime of WSNs