Delay-Optimal Buffer-Aware Probabilistic Scheduling with Adaptive Transmission

Cross-layer scheduling is a promising way to improve Quality of Service (QoS) given a power constraint. In this paper, we investigate the system with random data arrival and adaptive transmission. Probabilistic scheduling strategies aware of the buffer state are applied to generalize conventional deterministic scheduling. Based on this, the average delay and power consumption are analysed by Markov reward process. The optimal delay-power tradeoff curve is the Pareto frontier of the feasible delay-power region. It is proved that the optimal delay-power tradeoff is piecewise-linear, whose vertices are obtained by deterministic strategies. Moreover, the corresponding strategies of the optimal tradeoff curve are threshold-based, hence can be obtained by a proposed effective algorithm. On the other hand, we formulate a linear programming to minimize the average delay given a fixed power constraint. By varying the power constraint, the optimal delay-power tradeoff curve can also be obtained. It is demonstrated that the algorithm result and the optimization result match each other, and are further validated by Monte-Carlo simulation.Comment: 6 pages, 4 figures, accepted by IEEE ICCC 201

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs