What question did this study set out to answer?

The study aims to enhance load frequency control in islanded microgrids through adaptive PID tuning using reinforcement learning techniques.

May 7, 2026Open Access

Adaptive PID-based deep reinforcement learning for load frequency control in islanded microgrids with heterogeneous resources and energy storage

Key Points

The study aims to enhance load frequency control in islanded microgrids through adaptive PID tuning using reinforcement learning techniques.
Evaluated DDPG and TD3 reinforcement learning algorithms for tuning PID controllers.
Implemented offline training of an agent using a reward function for optimal control policy.
Assessed control performance in simulations of islanded microgrid systems.
DDPG-based PID controllers outperformed conventional PID controllers in dynamic response and robustness.
TD3-PID demonstrated better stability and reduced oscillations compared to DDPG-PID.
The RL-based adaptive PID controller provided improved overall performance in load demand tracking.

Abstract

Load frequency control (LFC) is an essential measure in maintaining stability in power systems in islanded microgrids that include heterogeneous generation sources and energy storage systems. Island microgrid systems (IMGs) that integrate renewable energy sources widely use proportional-integral-derivative (PID) controllers for LFC. However, the overall control performance is highly sensitive to the accurate tuning of PID controller parameters. To address this problem, this paper proposes an adaptive PID tuning approach that studies the individual evaluations of the Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning algorithms. The proposed reinforcement learning (RL)-based adaptive PID tuning approach is used to adaptively regulate PID gains under the inherent uncertainties of the IMGs environment. The proposed RL-PID controller approach employs an agent that is trained offline through repeated interactions with the IMGs model, where a suitable reward function guides the learning process toward an optimal control policy. Once trained, the agent is implemented online to continuously update the PID gains for coordinated control of the Wind Turbine Generator (WTG), Solar Photovoltaic (SPV), Fuel Cell (FC) units, Electric Vehicle (EV), and Biogas Turbine Generator (BTG), ensuring effective load demand tracking. Simulations of an IMGs demonstrate that both DDPG-based PID controllers and TD3-based PID controllers outperform conventional PID controllers in dynamic response, settling time, and robustness to disturbances. Furthermore, the TD3-PID shows better stability and reduced oscillations compared to the DDPG-PID, which can be explained by the fact that it improves the policy update mechanism, leading to more effective adjustments in response to system changes. Although studied PID tuning approaches incorporate intelligent mechanisms for gain adjustment, the results indicate that the RL-based adaptive PID controller provides improved overall performance.

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper