Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Uplink packet scheduling in cellular networks with relaying—comparative study

Deployment of intermediate relay nodes in cellular networks, e.g. UMTS/ HSPA, has been proposed for service enhancement, which is of particular importance for uplink users at the cell edge suffering from low power capacity and relatively poor channel conditions. In this paper, we propose and investigate a number of uplink packet scheduling schemes deploying the relay functionality in different ways. Using a combined packet and flow level analysis capturing the specifics of the scheduling schemes and the random behavior of the users (initiation and completion of flow transfers), the performance of the various schemes is evaluated and compared to a reference scenario where relaying is not used. The main performance measures considered in our study are realized data rates and mean flow transfer times. Interestingly, considering flow transfer times, it is found that the use of relay nodes is not only particularly beneficial for users at the cell edge but also has a strong, positive effect on the performance of users at other locations not transmitting via a relay node

Cooperative Relaying in Wireless Networks under Spatially and Temporally Correlated Interference

We analyze the performance of an interference-limited, decode-and-forward, cooperative relaying system that comprises a source, a destination, and $N$ relays, placed arbitrarily on the plane and suffering from interference by a set of interferers placed according to a spatial Poisson process. In each transmission attempt, first the transmitter sends a packet; subsequently, a single one of the relays that received the packet correctly, if such a relay exists, retransmits it. We consider both selection combining and maximal ratio combining at the destination, Rayleigh fading, and interferer mobility. We derive expressions for the probability that a single transmission attempt is successful, as well as for the distribution of the transmission attempts until a packet is transmitted successfully. Results provide design guidelines applicable to a wide range of systems. Overall, the temporal and spatial characteristics of the interference play a significant role in shaping the system performance. Maximal ratio combining is only helpful when relays are close to the destination; in harsh environments, having many relays is especially helpful, and relay placement is critical; the performance improves when interferer mobility increases; and a tradeoff exists between energy efficiency and throughput

A Cross-Layer Design Based on Geographic Information for Cooperative Wireless Networks

Most of geographic routing approaches in wireless ad hoc and sensor networks do not take into consideration the medium access control (MAC) and physical layers when designing a routing protocol. In this paper, we focus on a cross-layer framework design that exploits the synergies between network, MAC, and physical layers. In the proposed CoopGeo, we use a beaconless forwarding scheme where the next hop is selected through a contention process based on the geographic position of nodes. We optimize this Network-MAC layer interaction using a cooperative relaying technique with a relay selection scheme also based on geographic information in order to improve the system performance in terms of reliability.Comment: in 2010 IEEE 71st Vehicular Technology Conference, 201