Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Network Lifetime Maximization With Node Admission in Wireless Multimedia Sensor Networks

Wireless multimedia sensor networks (WMSNs) are expected to support multimedia services such as delivery of video and audio streams. However, due to the relatively stringent quality-of-service (QoS) requirements of multimedia services (e.g., high transmission rates and timely delivery) and the limited wireless resources, it is possible that not all the potential sensor nodes can be admitted into the network. Thus, node admission is essential for WMSNs, which is the target of this paper. Specifically, we aim at the node admission and its interaction with power allocation and link scheduling. A cross-layer design is presented as a two-stage optimization problem, where at the first stage the number of admitted sensor nodes is maximized, and at the second stage the network lifetime is maximized. Interestingly, it is proved that the two-stage optimization problem can be converted to a one-stage optimization problem with a more compact and concise mathematical form. Numerical results demonstrate the effectiveness of the two-stage and one-stage optimization frameworks

Cross-layer design of multi-hop wireless networks

MULTI -hop wireless networks are usually defined as a collection of nodes equipped with radio transmitters, which not only have the capability to communicate each other in a multi-hop fashion, but also to route each others’ data packets. The distributed nature of such networks makes them suitable for a variety of applications where there are no assumed reliable central entities, or controllers, and may significantly improve the scalability issues of conventional single-hop wireless networks. This Ph.D. dissertation mainly investigates two aspects of the research issues related to the efficient multi-hop wireless networks design, namely: (a) network protocols and (b) network management, both in cross-layer design paradigms to ensure the notion of service quality, such as quality of service (QoS) in wireless mesh networks (WMNs) for backhaul applications and quality of information (QoI) in wireless sensor networks (WSNs) for sensing tasks. Throughout the presentation of this Ph.D. dissertation, different network settings are used as illustrative examples, however the proposed algorithms, methodologies, protocols, and models are not restricted in the considered networks, but rather have wide applicability. First, this dissertation proposes a cross-layer design framework integrating a distributed proportional-fair scheduler and a QoS routing algorithm, while using WMNs as an illustrative example. The proposed approach has significant performance gain compared with other network protocols. Second, this dissertation proposes a generic admission control methodology for any packet network, wired and wireless, by modeling the network as a black box, and using a generic mathematical 0. Abstract 3 function and Taylor expansion to capture the admission impact. Third, this dissertation further enhances the previous designs by proposing a negotiation process, to bridge the applications’ service quality demands and the resource management, while using WSNs as an illustrative example. This approach allows the negotiation among different service classes and WSN resource allocations to reach the optimal operational status. Finally, the guarantees of the service quality are extended to the environment of multiple, disconnected, mobile subnetworks, where the question of how to maintain communications using dynamically controlled, unmanned data ferries is investigated

A Cross-Layer Design Based on Geographic Information for Cooperative Wireless Networks

Most of geographic routing approaches in wireless ad hoc and sensor networks do not take into consideration the medium access control (MAC) and physical layers when designing a routing protocol. In this paper, we focus on a cross-layer framework design that exploits the synergies between network, MAC, and physical layers. In the proposed CoopGeo, we use a beaconless forwarding scheme where the next hop is selected through a contention process based on the geographic position of nodes. We optimize this Network-MAC layer interaction using a cooperative relaying technique with a relay selection scheme also based on geographic information in order to improve the system performance in terms of reliability.Comment: in 2010 IEEE 71st Vehicular Technology Conference, 201