35,410 research outputs found
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
Emission-aware Energy Storage Scheduling for a Greener Grid
Reducing our reliance on carbon-intensive energy sources is vital for
reducing the carbon footprint of the electric grid. Although the grid is seeing
increasing deployments of clean, renewable sources of energy, a significant
portion of the grid demand is still met using traditional carbon-intensive
energy sources. In this paper, we study the problem of using energy storage
deployed in the grid to reduce the grid's carbon emissions. While energy
storage has previously been used for grid optimizations such as peak shaving
and smoothing intermittent sources, our insight is to use distributed storage
to enable utilities to reduce their reliance on their less efficient and most
carbon-intensive power plants and thereby reduce their overall emission
footprint. We formulate the problem of emission-aware scheduling of distributed
energy storage as an optimization problem, and use a robust optimization
approach that is well-suited for handling the uncertainty in load predictions,
especially in the presence of intermittent renewables such as solar and wind.
We evaluate our approach using a state of the art neural network load
forecasting technique and real load traces from a distribution grid with 1,341
homes. Our results show a reduction of >0.5 million kg in annual carbon
emissions -- equivalent to a drop of 23.3% in our electric grid emissions.Comment: 11 pages, 7 figure, This paper will appear in the Proceedings of the
ACM International Conference on Future Energy Systems (e-Energy 20) June
2020, Australi
Multiple Timescale Energy Scheduling for Wireless Communication with Energy Harvesting Devices
The primary challenge in wireless communication with energy harvesting devices is to efficiently utilize the harvesting energy such that the data packet transmission could be supported. This challenge stems from not only QoS requirement imposed by the wireless communication application, but also the energy harvesting dynamics and the limited battery capacity. Traditional solar predictable energy harvesting models are perturbed by prediction errors, which could deteriorate the energy management algorithms based on this models. To cope with these issues, we first propose in this paper a non-homogenous Markov chain model based on experimental data, which can accurately describe the solar energy harvesting process in contrast to traditional predictable energy models. Due to different timescale between the energy harvesting process and the wireless data transmission process, we propose a general framework of multiple timescale Markov decision process (MMDP) model to formulate the joint energy scheduling and transmission control problem under different timescales. We then derive the optimal control policies via a joint dynamic programming and value iteration approach. Extensive simulations are carried out to study the performances of the proposed schemes
- …