Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Energy-Efficient NOMA Enabled Heterogeneous Cloud Radio Access Networks

Heterogeneous cloud radio access networks (H-CRANs) are envisioned to be promising in the fifth generation (5G) wireless networks. H-CRANs enable users to enjoy diverse services with high energy efficiency, high spectral efficiency, and low-cost operation, which are achieved by using cloud computing and virtualization techniques. However, H-CRANs face many technical challenges due to massive user connectivity, increasingly severe spectrum scarcity and energy-constrained devices. These challenges may significantly decrease the quality of service of users if not properly tackled. Non-orthogonal multiple access (NOMA) schemes exploit non-orthogonal resources to provide services for multiple users and are receiving increasing attention for their potential of improving spectral and energy efficiency in 5G networks. In this article a framework for energy-efficient NOMA H-CRANs is presented. The enabling technologies for NOMA H-CRANs are surveyed. Challenges to implement these technologies and open issues are discussed. This article also presents the performance evaluation on energy efficiency of H-CRANs with NOMA.Comment: This work has been accepted by IEEE Network. Pages 18, Figure

Context-aware Cluster Based Device-to-Device Communication to Serve Machine Type Communications

Billions of Machine Type Communication (MTC) devices are foreseen to be deployed in next ten years and therefore potentially open a new market for next generation wireless network. However, MTC applications have different characteristics and requirements compared with the services provided by legacy cellular networks. For instance, an MTC device sporadically requires to transmit a small data packet containing information generated by sensors. At the same time, due to the massive deployment of MTC devices, it is inefficient to charge their batteries manually and thus a long battery life is required for MTC devices. In this sense, legacy networks designed to serve human-driven traffics in real time can not support MTC efficiently. In order to improve the availability and battery life of MTC devices, context-aware device-to-device (D2D) communication is exploited in this paper. By applying D2D communication, some MTC users can serve as relays for other MTC users who experience bad channel conditions. Moreover, signaling schemes are also designed to enable the collection of context information and support the proposed D2D communication scheme. Last but not least, a system level simulator is implemented to evaluate the system performance of the proposed technologies and a large performance gain is shown by the numerical results

Energy harvesting over Rician fading channel: A performance analysis for half-duplex bidirectional sensor networks under hardware impairments

In this paper, a rigorous analysis of the performance of time-switching energy harvesting strategy that is applied for a half-duplex bidirectional wireless sensor network with intermediate relay over a Rician fading channel is presented to provide the exact-form expressions of the outage probability, achievable throughput and the symbol-error-rate (SER) of the system under the hardware impairment condition. Using the proposed probabilistic models for wireless channels between mobile nodes as well as for the hardware noises, we derive the outage probability of the system, and then the throughput and SER can be obtained as a result. Both exact analysis and asymptotic analysis at high signal-power-to-noise-ratio regime are provided. Monte Carlo simulation is also conducted to verify the analysis. This work confirms the effectiveness of energy harvesting applied in wireless sensor networks over a Rician fading channel, and can provide an insightful understanding about the effect of various parameters on the system performance.Web of Science186art. no. 1781