Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Efficient Routing and Scheduling in Wireless Networks

The temporal and spatial variation in wireless channel conditions, node mobility make it challenging to design protocols for wireless networks. In this thesis, we design efficient routing and scheduling algorithms which adapt to changing network conditions caused by varying link quality or node mobility to improve user-level performance. We design and analyze routing protocols for static, mobile and heterogeneous wireless networks. We analyze the performance of opportunistic and cooperative forwarding in static mesh networks showing that opportunism outperforms cooperation; we identify interference as the main cause for mitigating the potential gains achievable with cooperative forwarding. For mobile networks, we quantitatively analyze the tradeoff between state information collection (sampling frequency and number of bits per sample) and power consumption for a fixed source-to-destination goodput constraint. For heterogeneous networks comprising of both static and mobile nodes, we propose a greedy algorithm (adaptive-flood) which dynamically classifies individual nodes as routers/flooders depending on network conditions and demonstrate that it achieves performance equivalent to, and in some cases significantly better than, that of network-wide routing or flooding alone. Last, we consider an application-level wireless streaming scenario where multiple clients are streaming different videos from a cellular base station. We design a greedy algorithm for efficiently scheduling multiple video streams from a base station to mobile clients so as to minimize the total number of application-playout stalls. We develop models for coarse timescale wireless channel variation to aid network and application-layer protocol design

Design and analysis of a beacon-less routing protocol for large volume content dissemination in vehicular ad hoc networks

Largevolumecontentdisseminationispursuedbythegrowingnumberofhighquality applications for Vehicular Ad hoc NETworks(VANETs), e.g., the live road surveillance service and the video-based overtaking assistant service. For the highly dynamical vehicular network topology, beacon-less routing protocols have been proven to be efficient in achieving a balance between the system performance and the control overhead. However, to the authors’ best knowledge, the routing design for large volume content has not been well considered in the previous work, which will introduce new challenges, e.g., the enhanced connectivity requirement for a radio link. In this paper, a link Lifetime-aware Beacon-less Routing Protocol (LBRP) is designed for large volume content delivery in VANETs. Each vehicle makes the forwarding decision based on the message header information and its current state, including the speed and position information. A semi-Markov process analytical model is proposed to evaluate the expected delay in constructing one routing path for LBRP. Simulations show that the proposed LBRP scheme outperforms the traditional dissemination protocols in providing a low end-to-end delay. The analytical model is shown to exhibit a good match on the delay estimation with Monte Carlo simulations, as well

Adaptive Network Coding Schemes for Satellite Communications

In this paper, we propose two novel physical layer aware adaptive network coding and coded modulation schemes for time variant channels. The proposed schemes have been applied to different satellite communications scenarios with different Round Trip Times (RTT). Compared to adaptive network coding, and classical non-adaptive network coding schemes for time variant channels, as benchmarks, the proposed schemes demonstrate that adaptation of packet transmission based on the channel variation and corresponding erasures allows for significant gains in terms of throughput, delay and energy efficiency. We shed light on the trade-off between energy efficiency and delay-throughput gains, demonstrating that conservative adaptive approaches that favors less transmission under high erasures, might cause higher delay and less throughput gains in comparison to non-conservative approaches that favor more transmission to account for high erasures.Comment: IEEE Advanced Satellite Multimedia Systems Conference and the 14th Signal Processing for Space Communications Workshop (ASMS/SPSC), 201

Flow Level QoE of Video Streaming in Wireless Networks

The Quality of Experience (QoE) of streaming service is often degraded by frequent playback interruptions. To mitigate the interruptions, the media player prefetches streaming contents before starting playback, at a cost of delay. We study the QoE of streaming from the perspective of flow dynamics. First, a framework is developed for QoE when streaming users join the network randomly and leave after downloading completion. We compute the distribution of prefetching delay using partial differential equations (PDEs), and the probability generating function of playout buffer starvations using ordinary differential equations (ODEs) for CBR streaming. Second, we extend our framework to characterize the throughput variation caused by opportunistic scheduling at the base station, and the playback variation of VBR streaming. Our study reveals that the flow dynamics is the fundamental reason of playback starvation. The QoE of streaming service is dominated by the first moments such as the average throughput of opportunistic scheduling and the mean playback rate. While the variances of throughput and playback rate have very limited impact on starvation behavior.Comment: 14 page

Scheduling of Multicast and Unicast Services under Limited Feedback by using Rateless Codes

Many opportunistic scheduling techniques are impractical because they require accurate channel state information (CSI) at the transmitter. In this paper, we investigate the scheduling of unicast and multicast services in a downlink network with a very limited amount of feedback information. Specifically, unicast users send imperfect (or no) CSI and infrequent acknowledgements (ACKs) to a base station, and multicast users only report infrequent ACKs to avoid feedback implosion. We consider the use of physical-layer rateless codes, which not only combats channel uncertainty, but also reduces the overhead of ACK feedback. A joint scheduling and power allocation scheme is developed to realize multiuser diversity gain for unicast service and multicast gain for multicast service. We prove that our scheme achieves a near-optimal throughput region. Our simulation results show that our scheme significantly improves the network throughput over schemes employing fixed-rate codes or using only unicast communications