Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Wireless Power Transfer and Data Collection in Wireless Sensor Networks

In a rechargeable wireless sensor network, the data packets are generated by sensor nodes at a specific data rate, and transmitted to a base station. Moreover, the base station transfers power to the nodes by using Wireless Power Transfer (WPT) to extend their battery life. However, inadequately scheduling WPT and data collection causes some of the nodes to drain their battery and have their data buffer overflow, while the other nodes waste their harvested energy, which is more than they need to transmit their packets. In this paper, we investigate a novel optimal scheduling strategy, called EHMDP, aiming to minimize data packet loss from a network of sensor nodes in terms of the nodes' energy consumption and data queue state information. The scheduling problem is first formulated by a centralized MDP model, assuming that the complete states of each node are well known by the base station. This presents the upper bound of the data that can be collected in a rechargeable wireless sensor network. Next, we relax the assumption of the availability of full state information so that the data transmission and WPT can be semi-decentralized. The simulation results show that, in terms of network throughput and packet loss rate, the proposed algorithm significantly improves the network performance.Comment: 30 pages, 8 figures, accepted to IEEE Transactions on Vehicular Technolog

A Non-Cooperative Game Theoretical Approach For Power Control In Virtual MIMO Wireless Sensor Network

Power management is one of the vital issue in wireless sensor networks, where the lifetime of the network relies on battery powered nodes. Transmitting at high power reduces the lifetime of both the nodes and the network. One efficient way of power management is to control the power at which the nodes transmit. In this paper, a virtual multiple input multiple output wireless sensor network (VMIMO-WSN)communication architecture is considered and the power control of sensor nodes based on the approach of game theory is formulated. The use of game theory has proliferated, with a broad range of applications in wireless sensor networking. Approaches from game theory can be used to optimize node level as well as network wide performance. The game here is categorized as an incomplete information game, in which the nodes do not have complete information about the strategies taken by other nodes. For virtual multiple input multiple output wireless sensor network architecture considered, the Nash equilibrium is used to decide the optimal power level at which a node needs to transmit, to maximize its utility. Outcome shows that the game theoretic approach considered for VMIMO-WSN architecture achieves the best utility, by consuming less power.Comment: 12 pages, 8 figure

A Comprehensive Survey of Potential Game Approaches to Wireless Networks

Potential games form a class of non-cooperative games where unilateral improvement dynamics are guaranteed to converge in many practical cases. The potential game approach has been applied to a wide range of wireless network problems, particularly to a variety of channel assignment problems. In this paper, the properties of potential games are introduced, and games in wireless networks that have been proven to be potential games are comprehensively discussed.Comment: 44 pages, 6 figures, to appear in IEICE Transactions on Communications, vol. E98-B, no. 9, Sept. 201

Towards Optimal Distributed Node Scheduling in a Multihop Wireless Network through Local Voting

In a multihop wireless network, it is crucial but challenging to schedule transmissions in an efficient and fair manner. In this paper, a novel distributed node scheduling algorithm, called Local Voting, is proposed. This algorithm tries to semi-equalize the load (defined as the ratio of the queue length over the number of allocated slots) through slot reallocation based on local information exchange. The algorithm stems from the finding that the shortest delivery time or delay is obtained when the load is semi-equalized throughout the network. In addition, we prove that, with Local Voting, the network system converges asymptotically towards the optimal scheduling. Moreover, through extensive simulations, the performance of Local Voting is further investigated in comparison with several representative scheduling algorithms from the literature. Simulation results show that the proposed algorithm achieves better performance than the other distributed algorithms in terms of average delay, maximum delay, and fairness. Despite being distributed, the performance of Local Voting is also found to be very close to a centralized algorithm that is deemed to have the optimal performance

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig