79,405 research outputs found
Approximate Policy Iteration (API) with neural networks for the generalized single node energy storage problem
Energy storage problems are hard sequential decision-making problems often modelled as markov decision processes. Exact solution using dynamic programming quickly becomes implausible with large state spaces hence approximate dynamic programming using policy iteration (API) is often employed in such cases. API does not always work, one reason being that the approximation architectures used are often linear for computational tractability reasons. We propose a mathematical model which allows easier implementation of non-linear approximations with API. We use neural networks along with monte-carlo simulation to predict the future values for the generated states during the improvement step of the API algorithm. Our initial experiments suggest that the proposed method provides good results which can be further improved with more fine tuning of the neural network parameters
Local Water Storage Control for the Developing World
Most cities in India do not have water distribution networks that provide
water throughout the entire day. As a result, it is common for homes and
apartment buildings to utilize water storage systems that are filled during a
small window of time in the day when the water distribution network is active.
However, these water storage systems do not have disinfection capabilities, and
so long durations of storage (i.e., as few as four days) of the same water
leads to substantial increases in the amount of bacteria and viruses in that
water. This paper considers the stochastic control problem of deciding how much
water to store each day in the system, as well as deciding when to completely
empty the water system, in order to tradeoff: the financial costs of the water,
the health costs implicit in long durations of storing the same water, the
potential for a shortfall in the quantity of stored versus demanded water, and
water wastage from emptying the system. To solve this problem, we develop a new
Binary Dynamic Search (BiDS) algorithm that is able to use binary search in one
dimension to compute the value function of stochastic optimal control problems
with controlled resets to a single state and with constraints on the maximum
time span in between resets of the system
A probabilistic numerical method for optimal multiple switching problems in high dimension
In this paper, we present a probabilistic numerical algorithm combining dynamic programming, Monte Carlo simulations and local basis regressions to solve non-stationary optimal multiple switching problems in infinite horizon. We provide the rate of convergence of the method in terms of the time step used to discretize the problem, of the regression basis used to approximate conditional expectations, and of the truncating time horizon. To make the method viable for problems in high dimension and long time horizon, we extend a memory reduction method to the general Euler scheme, so that, when performing the numerical resolution, the storage of the Monte Carlo simulation paths is not needed. Then, we apply this algorithm to a model of optimal investment in power plants in dimension eight, i.e. with two different technologies and six random factors
Optimal Hour-Ahead Bidding in the Real-Time Electricity Market with Battery Storage using Approximate Dynamic Programming
There is growing interest in the use of grid-level storage to smooth
variations in supply that are likely to arise with increased use of wind and
solar energy. Energy arbitrage, the process of buying, storing, and selling
electricity to exploit variations in electricity spot prices, is becoming an
important way of paying for expensive investments into grid-level storage.
Independent system operators such as the NYISO (New York Independent System
Operator) require that battery storage operators place bids into an hour-ahead
market (although settlements may occur in increments as small as 5 minutes,
which is considered near "real-time"). The operator has to place these bids
without knowing the energy level in the battery at the beginning of the hour,
while simultaneously accounting for the value of leftover energy at the end of
the hour. The problem is formulated as a dynamic program. We describe and
employ a convergent approximate dynamic programming (ADP) algorithm that
exploits monotonicity of the value function to find a revenue-generating
bidding policy; using optimal benchmarks, we empirically show the computational
benefits of the algorithm. Furthermore, we propose a distribution-free variant
of the ADP algorithm that does not require any knowledge of the distribution of
the price process (and makes no assumptions regarding a specific real-time
price model). We demonstrate that a policy trained on historical real-time
price data from the NYISO using this distribution-free approach is indeed
effective.Comment: 28 pages, 11 figure
- …