Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Millimeter Wave Cellular Networks: A MAC Layer Perspective

The millimeter wave (mmWave) frequency band is seen as a key enabler of multi-gigabit wireless access in future cellular networks. In order to overcome the propagation challenges, mmWave systems use a large number of antenna elements both at the base station and at the user equipment, which lead to high directivity gains, fully-directional communications, and possible noise-limited operations. The fundamental differences between mmWave networks and traditional ones challenge the classical design constraints, objectives, and available degrees of freedom. This paper addresses the implications that highly directional communication has on the design of an efficient medium access control (MAC) layer. The paper discusses key MAC layer issues, such as synchronization, random access, handover, channelization, interference management, scheduling, and association. The paper provides an integrated view on MAC layer issues for cellular networks, identifies new challenges and tradeoffs, and provides novel insights and solution approaches.Comment: 21 pages, 9 figures, 2 tables, to appear in IEEE Transactions on Communication