Reinforcement Learning Applied to Multi Agent Modelling, the Case of the Iranian Power Market

Abstract

With increasing competition in the wholesale Electricity markets and advances in behavioral economics in recent years, the multi-agent modeling approach has been applied widely to simulate the outcome of the markets. The electricity market consists of power generating agents that compete over production in daily auction conducted by an independent system operator (ISO). The market clearing mechanism can be seen as a static game that repeats every hour. In this game, an agent proposes her price for the next day and the ISO chooses the best proposals that minimizes the total costs given the demand and the technical constraints. Agents are also assumed to learn from the outcomes and adjust their biding strategy accordingly. In this paper, we develop an agent-based model for the day-ahead and pay-as-bid electricity market in Iran. The objective is to compare the outcome of the market measured by the agents profit and the time to converge using three different strategies: greedy, random and reinforcement learning. The simulation results indicate that the reinforcement learning leads to higher profits with a faster convergence rate than the other two strategies