Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Two-Hop Connectivity to the Roadside in a VANET Under the Random Connection Model

We compute the expected number of cars that have at least one two-hop path to a fixed roadside unit in a one-dimensional vehicular ad hoc network in which other cars can be used as relays to reach a roadside unit when they do not have a reliable direct link. The pairwise channels between cars experience Rayleigh fading in the random connection model, and so exist, with probability function of the mutual distance between the cars, or between the cars and the roadside unit. We derive exact equivalents for this expected number of cars when the car density $\rho$ tends to zero and to infinity, and determine its behaviour using an infinite oscillating power series in $\rho$, which is accurate for all regimes. We also corroborate those findings to a realistic situation, using snapshots of actual traffic data. Finally, a normal approximation is discussed for the probability mass function of the number of cars with a two-hop connection to the origin. The probability mass function appears to be well fitted by a Gaussian approximation with mean equal to the expected number of cars with two hops to the origin.Comment: 21 pages, 7 figure

Impact analysis of the shortest path movement model on routing strategies for VDTNs in a rural region

Vehicular Delay-Tolerant Network (VDTN) appears as a particular application of the Delay-Tolerant Network (DTN) concept to transit networks. In this paper we analyze the use of a VDTN to provide asynchronous Internet access on a rural remote region scenario. Through simulation we evaluate the impact of a shortest path based movement model on the performance of four DTN routing protocols in respect to message delivery probability and message average delay.Part of this work has been supported by the Instituto de Telecomunicações, Portugal, in the framework of the Project VDTN@Lab, and by the Euro-NF Network of Excellence of Seven Framework Programme of EU

Research on Wireless Multi-hop Networks: Current State and Challenges

Wireless multi-hop networks, in various forms and under various names, are being increasingly used in military and civilian applications. Studying connectivity and capacity of these networks is an important problem. The scaling behavior of connectivity and capacity when the network becomes sufficiently large is of particular interest. In this position paper, we briefly overview recent development and discuss research challenges and opportunities in the area, with a focus on the network connectivity.Comment: invited position paper to International Conference on Computing, Networking and Communications, Hawaii, USA, 201

On the Benefits of Network-Level Cooperation in Millimeter-Wave Communications

Relaying techniques for millimeter-wave wireless networks represent a powerful solution for improving the transmission performance. In this work, we quantify the benefits in terms of delay and throughput for a random-access multi-user millimeter-wave wireless network, assisted by a full-duplex network cooperative relay. The relay is equipped with a queue for which we analyze the performance characteristics (e.g., arrival rate, service rate, average size, and stability condition). Moreover, we study two possible transmission schemes: fully directional and broadcast. In the former, the source nodes transmit a packet either to the relay or to the destination by using narrow beams, whereas, in the latter, the nodes transmit to both the destination and the relay in the same timeslot by using a wider beam, but with lower beamforming gain. In our analysis, we also take into account the beam alignment phase that occurs every time a transmitter node changes the destination node. We show how the beam alignment duration, as well as position and number of transmitting nodes, significantly affect the network performance. Moreover, we illustrate the optimal transmission scheme (i.e., broadcast or fully directional) for several system parameters and show that a fully directional transmission is not always beneficial, but, in some scenarios, broadcasting and relaying can improve the performance in terms of throughput and delay.Comment: arXiv admin note: text overlap with arXiv:1804.0945

Improving vehicular delay-tolerant network performance with relay nodes

“Copyright © [2009] IEEE. Reprinted from Next Generation Internet Network. NGI '09). ISBN:978-1-4244-4244-7. This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.”Vehicular Delay-Tolerant Networking (VDTN) is an extension of the Delay-Tolerant Network (DTN) architecture concept to transit networks. VDTN architecture handles non-real time applications, exploiting vehicles to enable connectivity under unreliable scenarios with unstable links and where an end-to-end path may not exist. Intuitively, the use of stationary store-and-forward devices (relay nodes) located at crossroads where vehicles meet them and should improve the message delivery probability. In this paper, we analyze the influence of the number of relay nodes, in urban scenarios with different numbers of vehicles. It was shown that relay nodes significantly improve the message delivery probability on studied DTN routing protocols.Part of this work has been supported by the Instituto de Telecomunicações, Next Generation Networks and Applications Group, Portugal, in the framework of the Project VDTN@Lab, and by the Euro-NF Network of Excellence of Seven Framework Programme of EU