Blind Decision Making: Reinforcement Learning with Delayed Observations

Reinforcement learning typically assumes that the state update from the previous actions happens instantaneously, and thus can be used for making future decisions. However, this may not always be true. When the state update is not available, the decision taken is partly in the blind since it cannot rely on the current state information. This paper proposes an approach, where the delay in the knowledge of the state can be used, and the decisions are made based on the available information which may not include the current state information. One approach could be to include the actions after the last-known state as a part of the state information, however, that leads to an increased state-space making the problem complex and slower in convergence. The proposed algorithm gives an alternate approach where the state space is not enlarged, as compared to the case when there is no delay in the state update. Evaluations on the basic RL environments further illustrate the improved performance of the proposed algorithm

Secondary frequency regulation scheme based on improved virtual synchronous generator in an islanded microgrid

To aim at the defects of the traditional VSG (virtual synchronous generator) control schemes to only complete the primary frequency regulation equivalent to droop control, and not to realize the secondary frequency regulation, this paper proposes a new control scheme to realize the secondary frequency regulation of the VSG, such that system frequency can return to the rated value. In this method, the torque change process of the rotor of synchronous generator is firstly analyzed when the system active power fluctuates. And then, a novel secondary frequency regulation control strategy of the VSG is proposed, where the small-signal model of power transmission of the VSG is established and transfer function is analyzed, and parameters design and stability analysis of the control system are conducted. Finally, simulation experiments show the correctness and effectiveness of the proposed method

Effect of Wireless Communication Delay on DC Microgrids Performance

This paper investigates the effect of wireless communication technologies latency on the converters and the bus voltage of centrally communication based controlled DC microgrids (MGs) during islanding. A DC microgrid with its communication based control scheme was modeled to show the impact of latency. Simulation results show that the impact may be severe depending on the design, and the operational condition of the microgrid before latency occurs

Local frequency restoration for droop-controlled parallel inverters in islanded microgrids

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksIn islanded microgrids, voltage source inverters working in parallel are expected to provide regulation of the local frequency while granting active power sharing. This paper presents a local control approach at each inverter based on an event-driven operation of a parameter-varying filter. It ensures perfect active power sharing and controllable accuracy for frequency restoration without requiring the exchange of control data between inverters over the communication network. The paper includes stability analysis and design guidelines for the control parameters using a modeling approach that considers the interaction between inverters. Selected experimental results on a three-inverter laboratory microgrid corroborate the effectiveness of the proposed control scheme, and outlines its advantages with respect to previous similar schemes and the performance cost that implies not using communicationsPeer ReviewedPostprint (author's final draft