Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Cooperative medium access control based on spectrum leasing

Based on cooperative spectrum leasing, a distributed “win–win” (WW) cooperative framework is designed to encourage the licensed source node (SN) to lease some part of its spectral resources to the unlicensed relay node (RN) for the sake of simultaneously improving the SN’s achievable rate and for reducing the energy consumption (EC). The potential candidate RNs carry out autonomous decisions concerning whether to contend for a cooperative transmission opportunity, which could dissipate some of their battery power, while conveying their traffic in light of their individual service requirements. Furthermore, a WW cooperative medium-access-control (MAC) protocol is designed to implement the proposed distributed WW cooperative framework. Simulation results demonstrate that our WW cooperative MAC protocol is capable of providing both substantial rate improvements and considerable energy savings for the cooperative spectrum leasing system

Applications of Repeated Games in Wireless Networks: A Survey

A repeated game is an effective tool to model interactions and conflicts for players aiming to achieve their objectives in a long-term basis. Contrary to static noncooperative games that model an interaction among players in only one period, in repeated games, interactions of players repeat for multiple periods; and thus the players become aware of other players' past behaviors and their future benefits, and will adapt their behavior accordingly. In wireless networks, conflicts among wireless nodes can lead to selfish behaviors, resulting in poor network performances and detrimental individual payoffs. In this paper, we survey the applications of repeated games in different wireless networks. The main goal is to demonstrate the use of repeated games to encourage wireless nodes to cooperate, thereby improving network performances and avoiding network disruption due to selfish behaviors. Furthermore, various problems in wireless networks and variations of repeated game models together with the corresponding solutions are discussed in this survey. Finally, we outline some open issues and future research directions.Comment: 32 pages, 15 figures, 5 tables, 168 reference

Energy efficiency of some non-cooperative, cooperative and hybrid communication schemes in multi-relay WSNs

In this paper we analyze the energy efficiency of single-hop, multi-hop, cooperative selective decode-and-forward, cooperative incremental decode-and-forward, and even the combination of cooperative and non-cooperative schemes, in wireless sensor networks composed of several nodes. We assume that, as the sensor nodes can experience either non line-of-sight or some line-of-sight conditions, the Nakagami-m fading distribution is used to model the wireless environment. The energy efficiency analysis is constrained by a target outage probability and an end-to-end throughput. Our results show that in most scenarios cooperative incremental schemes are more energy efficient than the other methods

An energy-efficient spectrum-aware reinforcement learning-based clustering algorithm for cognitive radio sensor networks

It is well-known that clustering partitions network into logical groups of nodes in order to achieve energy efficiency and to enhance dynamic channel access in cognitive radio through cooperative sensing. While the topic of energy efficiency has been well investigated in conventional wireless sensor networks, the latter has not been extensively explored. In this paper, we propose a reinforcement learning-based spectrum-aware clustering algorithm that allows a member node to learn the energy and cooperative sensing costs for neighboring clusters to achieve an optimal solution. Each member node selects an optimal cluster that satisfies pairwise constraints, minimizes network energy consumption and enhances channel sensing performance through an exploration technique. We first model the network energy consumption and then determine the optimal number of clusters for the network. The problem of selecting an optimal cluster is formulated as a Markov Decision Process (MDP) in the algorithm and the obtained simulation results show convergence, learning and adaptability of the algorithm to dynamic environment towards achieving an optimal solution. Performance comparisons of our algorithm with the Groupwise Spectrum Aware (GWSA)-based algorithm in terms of Sum of Square Error (SSE), complexity, network energy consumption and probability of detection indicate improved performance from the proposed approach. The results further reveal that an energy savings of 9% and a significant Primary User (PU) detection improvement can be achieved with the proposed approach

An Auction-based Mechanism for Cooperative Sensing in Cognitive Networks

International audienceIn this paper, we propose an auction-based cooperative sensing protocol for secondary users in cognitive networks. The proposed auction mechanism is based on a novel modified Vickrey auction with a three dimensional bid, that accounts for detection gains as well as for virtual currency gains. We present a formal proof to show that the proposed three dimensional bidding mechanism preserves the truthfulness property of the classic Vickrey auction. The cooperative auction is combined with a prioritized access scheme to increase the efficiency and to reduce the response time for the coalition formation procedure. Our auction-based cooperative sensing mechanism can be easily applied to different network scenarios, by defining specific utility functions. The proposed cooperative sensing auctioning mechanism is illustrated for both downlink and uplink. Our simulation results show that users' cooperation is incentivized by the proposed algorithm, which leads to significant detection gains for the downlink and the uplink scenarios, with a more efficient energy expenditure