3,348 research outputs found
Decentralized Delay Optimal Control for Interference Networks with Limited Renewable Energy Storage
In this paper, we consider delay minimization for interference networks with
renewable energy source, where the transmission power of a node comes from both
the conventional utility power (AC power) and the renewable energy source. We
assume the transmission power of each node is a function of the local channel
state, local data queue state and local energy queue state only. In turn, we
consider two delay optimization formulations, namely the decentralized
partially observable Markov decision process (DEC-POMDP) and Non-cooperative
partially observable stochastic game (POSG). In DEC-POMDP formulation, we
derive a decentralized online learning algorithm to determine the control
actions and Lagrangian multipliers (LMs) simultaneously, based on the policy
gradient approach. Under some mild technical conditions, the proposed
decentralized policy gradient algorithm converges almost surely to a local
optimal solution. On the other hand, in the non-cooperative POSG formulation,
the transmitter nodes are non-cooperative. We extend the decentralized policy
gradient solution and establish the technical proof for almost-sure convergence
of the learning algorithms. In both cases, the solutions are very robust to
model variations. Finally, the delay performance of the proposed solutions are
compared with conventional baseline schemes for interference networks and it is
illustrated that substantial delay performance gain and energy savings can be
achieved
On the Effects of Battery Imperfections in an Energy Harvesting Device
Energy Harvesting allows the devices in a Wireless Sensor Network to recharge
their batteries through environmental energy sources. While in the literature
the main focus is on devices with ideal batteries, in reality several
inefficiencies have to be considered to correctly design the operating regimes
of an Energy Harvesting Device (EHD). In this work we describe how the
throughput optimization problem changes under \emph{real battery} constraints
in an EHD. In particular, we consider imperfect knowledge of the state of
charge of the battery and storage inefficiencies, \emph{i.e.}, part of the
harvested energy is wasted in the battery recharging process. We formulate the
problem as a Markov Decision Process, basing our model on some realistic
observations about transmission, consumption and harvesting power. We find the
performance upper bound with a real battery and numerically discuss the novelty
introduced by the real battery effects. We show that using the \emph{old}
policies obtained without considering the real battery effects is strongly
sub-optimal and may even result in zero throughput.Comment: In Proc. IEEE International Conference on Computing, Networking and
Communications, pp. 942-948, Feb. 201
Joint Transmission and Energy Transfer Policies for Energy Harvesting Devices with Finite Batteries
One of the main concerns in traditional Wireless Sensor Networks (WSNs) is
energy efficiency. In this work, we analyze two techniques that can extend
network lifetime. The first is Ambient \emph{Energy Harvesting} (EH), i.e., the
capability of the devices to gather energy from the environment, whereas the
second is Wireless \emph{Energy Transfer} (ET), that can be used to exchange
energy among devices. We study the combination of these techniques, showing
that they can be used jointly to improve the system performance. We consider a
transmitter-receiver pair, showing how the ET improvement depends upon the
statistics of the energy arrivals and the energy consumption of the devices.
With the aim of maximizing a reward function, e.g., the average transmission
rate, we find performance upper bounds with and without ET, define both online
and offline optimization problems, and present results based on realistic
energy arrivals in indoor and outdoor environments. We show that ET can
significantly improve the system performance even when a sizable fraction of
the transmitted energy is wasted and that, in some scenarios, the online
approach can obtain close to optimal performance.Comment: 16 pages, 12 figure
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
- …