20,436 research outputs found
Inverter PQ Control With Trajectory Tracking Capability For Microgrids Based On Physics-informed Reinforcement Learning
The increasing penetration of inverter-based resources (IBRs) calls for an advanced active and reactive power (PQ) control strategy in microgrids. To enhance the controllability and flexibility of the IBRs, this paper proposed an adaptive PQ control method with trajectory tracking capability, combining model-based analysis, physics-informed reinforcement learning (RL), and power hardware-in-the-loop (HIL) experiments. First, model-based analysis proves that there exists an adaptive proportional-integral controller with time-varying gains that can ensure any exponential PQ output trajectory of IBRs. These gains consist of a constant factor and an exponentially decaying factor, which are then obtained using a model-free deep reinforcement learning approach known as the twin delayed deeper deterministic policy gradient. With the model-based derivation, the learning space of the RL agent is narrowed down from a function space to a real space, which reduces the training complexity significantly. Finally, the proposed method is verified through numerical simulation in MATLAB-Simulink and power HIL experiments in the CURENT center.With the physics-informed learning method, exponential response time constants can be freely assigned to IBRs, and they can follow any predefined trajectory without complicated gain tuning
Reinforcement Learning for UAV Attitude Control
Autopilot systems are typically composed of an "inner loop" providing
stability and control, while an "outer loop" is responsible for mission-level
objectives, e.g. way-point navigation. Autopilot systems for UAVs are
predominately implemented using Proportional, Integral Derivative (PID) control
systems, which have demonstrated exceptional performance in stable
environments. However more sophisticated control is required to operate in
unpredictable, and harsh environments. Intelligent flight control systems is an
active area of research addressing limitations of PID control most recently
through the use of reinforcement learning (RL) which has had success in other
applications such as robotics. However previous work has focused primarily on
using RL at the mission-level controller. In this work, we investigate the
performance and accuracy of the inner control loop providing attitude control
when using intelligent flight control systems trained with the state-of-the-art
RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy
Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate
these unknowns we first developed an open-source high-fidelity simulation
environment to train a flight controller attitude control of a quadrotor
through RL. We then use our environment to compare their performance to that of
a PID controller to identify if using RL is appropriate in high-precision,
time-critical flight control.Comment: 13 pages, 9 figure
- …