12,582 research outputs found
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs
We present multi-agent A* (MAA*), the first complete and optimal heuristic
search algorithm for solving decentralized partially-observable Markov decision
problems (DEC-POMDPs) with finite horizon. The algorithm is suitable for
computing optimal plans for a cooperative group of agents that operate in a
stochastic environment such as multirobot coordination, network traffic
control, `or distributed resource allocation. Solving such problems efiectively
is a major challenge in the area of planning under uncertainty. Our solution is
based on a synthesis of classical heuristic search and decentralized control
theory. Experimental results show that MAA* has significant advantages. We
introduce an anytime variant of MAA* and conclude with a discussion of
promising extensions such as an approach to solving infinite horizon problems.Comment: Appears in Proceedings of the Twenty-First Conference on Uncertainty
in Artificial Intelligence (UAI2005
Queue-Aware Distributive Resource Control for Delay-Sensitive Two-Hop MIMO Cooperative Systems
In this paper, we consider a queue-aware distributive resource control
algorithm for two-hop MIMO cooperative systems. We shall illustrate that relay
buffering is an effective way to reduce the intrinsic half-duplex penalty in
cooperative systems. The complex interactions of the queues at the source node
and the relays are modeled as an average-cost infinite horizon Markov Decision
Process (MDP). The traditional approach solving this MDP problem involves
centralized control with huge complexity. To obtain a distributive and low
complexity solution, we introduce a linear structure which approximates the
value function of the associated Bellman equation by the sum of per-node value
functions. We derive a distributive two-stage two-winner auction-based control
policy which is a function of the local CSI and local QSI only. Furthermore, to
estimate the best fit approximation parameter, we propose a distributive online
stochastic learning algorithm using stochastic approximation theory. Finally,
we establish technical conditions for almost-sure convergence and show that
under heavy traffic, the proposed low complexity distributive control is global
optimal.Comment: 30 pages, 7 figure
- …