10,296 research outputs found
DYNAMIC PROGRAMMING: HAS ITS DAY ARRIVED?
Research Methods/ Statistical Methods,
Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems
In many Cyber-Physical Systems, we encounter the problem of remote state
estimation of geographically distributed and remote physical processes. This
paper studies the scheduling of sensor transmissions to estimate the states of
multiple remote, dynamic processes. Information from the different sensors have
to be transmitted to a central gateway over a wireless network for monitoring
purposes, where typically fewer wireless channels are available than there are
processes to be monitored. For effective estimation at the gateway, the sensors
need to be scheduled appropriately, i.e., at each time instant one needs to
decide which sensors have network access and which ones do not. To address this
scheduling problem, we formulate an associated Markov decision process (MDP).
This MDP is then solved using a Deep Q-Network, a recent deep reinforcement
learning algorithm that is at once scalable and model-free. We compare our
scheduling algorithm to popular scheduling algorithms such as round-robin and
reduced-waiting-time, among others. Our algorithm is shown to significantly
outperform these algorithms for many example scenarios
Deterministic Equations for Stochastic Spatial Evolutionary Games
Spatial evolutionary games model individuals who are distributed in a spatial
domain and update their strategies upon playing a normal form game with their
neighbors. We derive integro-differential equations as deterministic
approximations of the microscopic updating stochastic processes. This
generalizes the known mean-field ordinary differential equations and provide a
powerful tool to investigate the spatial effects in populations evolution. The
deterministic equations allow to identify many interesting features of the
evolution of strategy profiles in a population, such as standing and traveling
waves, and pattern formation, especially in replicator-type evolutions
Optimal delivery strategies for heterogeneous groups of porkers
Farms;Strategic Planning;organization and management
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
- …