PHASE COMPENSATION ACTIVE DISTURBANCE REjECTION CONTROL STRATEGY BASED ON DEEP REINFORCEMENT LEARNING

Li Suyang; Shao Baofu; Zhou Xuesong; Ma Youjie; Tao Long; Wen Hulong

doi:10.19912/j.0254-0096.tynxb.2024-1572

PDF(2518 KB)

Acta Energiae Solaris Sinica ›› 2026, Vol. 47 ›› Issue (1) : 288-300. DOI: 10.19912/j.0254-0096.tynxb.2024-1572

PHASE COMPENSATION ACTIVE DISTURBANCE REjECTION CONTROL STRATEGY BASED ON DEEP REINFORCEMENT LEARNING

Li Suyang¹, Shao Baofu², Zhou Xuesong¹, Ma Youjie¹, Tao Long¹, Wen Hulong³

Author information +

History +

Abstract

To address the phase lag issue in bus voltage control of DC microgrids, a deep reinforcement learning-based active disturbance rejection control strategy incorporating a phase compensation network is developed for bidirectional DC-DC converters in energy storage systems. In this strategy,the order of the linear extended state observer is reduced, and a phase compensation network is connected in series with its total disturbance channel to provide the required lead phase angle. Subsequently,the mechanism by which the compensation link enhances disturbance suppression performance is analyzed,and the compensation parameter ranges are configured. Besides,a control parameter optimization model is established based on deep reinforcement learning algorithms. By training an intelligent agent to interact with the DC microgrid environment,the optimal combination of policy parameters is explored to achieve adaptive adjustment of observer bandwidth and correction parameters. Finally,simulations are conducted to compare the tracking ability and control accuracy of different control methods for bus voltage disturbances under typical operating conditions,and the results confirm the effectiveness of the proposed control strategy.

Key words

energy storage / microgrids / power converters / deep reinforcement learning / Markov processes / disturbance rejection

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Li Suyang, Shao Baofu, Zhou Xuesong, Ma Youjie, Tao Long, Wen Hulong. PHASE COMPENSATION ACTIVE DISTURBANCE REjECTION CONTROL STRATEGY BASED ON DEEP REINFORCEMENT LEARNING[J]. Acta Energiae Solaris Sinica. 2026, 47(1): 288-300 https://doi.org/10.19912/j.0254-0096.tynxb.2024-1572

References

[1] 张伟亮, 张辉, 支娜, 等. 基于节点源荷电流差分的直流微电网储能变换器控制策略[J]. 电工技术学报, 2022, 37(9): 2199-2210.
ZHANG W L, ZHANG H, ZHI N, et al.Control strategy of DC microgrid energy storage converter based on node differential current[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2199-2210.
[2] 薛花, 胡英俊, 董丙伟, 等. 基于分层控制的光-储-燃直流供电系统能量管理方法[J]. 电力系统自动化, 2018, 42(21): 53-62.
XUE H, HU Y J, DONG B W, et al.Hierarchical control based energy management method for photovoltaic-energy storage-fuel cell DC generation system[J]. Automation of electric power systems, 2018, 42(21): 53-62.
[3] 代丽, 王君瑞, 谭露, 等. 基于自适应下垂控制的光储直流微网控制策略研究[J]. 太阳能学报, 2024, 45(8): 154-163.
DAI L, WANG J R, TAN L, et al.Research on control strategy of PV storage DC microgrid based on adaptive droop control[J]. Acta energiae solaris sinica, 2024, 45(8): 154-163.
[4] 曾国辉, 朱相臣, 曾志伟, 等. 具有公共低压直流母线电压支撑功能的储能单元SOC自动均衡控制策略[J]. 中国电机工程学报, 2022, 42(19): 7160-7170.
ZENG G H, ZHU X C, ZENG Z W, et al.Energy storage unit state-of-charge automatic equalization control strategy with common LVDC bus voltage support function[J]. Proceedings of the CSEE, 2022, 42(19): 7160-7170.
[5] THOMAS T, MISHRA M K, KUMAR C, et al.Control of a PV-wind based DC microgrid with hybrid energy storage system using Lyapunov approach and sliding mode control[J]. IEEE transactions on industry applications, 2024, 60(2): 3746-3758.
[6] 马幼捷, 杨清, 周雪松, 等. 基于模糊神经网络的储能侧双向DC-DC变换器自抗扰控制策略[J]. 太阳能学报, 2023, 44(10): 488-495.
MA Y J, YANG Q, ZHOU X S, et al.Active disturbance rejection control strategy of bidirectional DC-DC converter on energy storage side based on fuzzy neural network[J]. Acta energiae solaris sinica, 2023, 44(10): 488-495.
[7] 张世欣, 皇金锋, 杨艺. 基于平坦理论的直流微电网双向DC-DC变换器改进滑模自抗扰控制[J]. 电力系统保护与控制, 2023, 51(5): 107-116.
ZHANG S X, HUANG J F, YANG Y.Improved sliding mode and active disturbance rejection control based on flatness theory for a bi-directional DC-DC converter in a DC microgrid[J]. Power system protection and control, 2023, 51(5): 107-116.
[8] GAO Z Q.Scaling and bandwidth-parameterization based controller tuning[C]//Proceedings of the 2003 American Control Conference, 2003. Denver, CO, USA, 2003: 4989-4996.
[9] 高钦和, 董家臣. 扩张状态观测器的观测误差前馈补偿设计[J]. 国防科技大学学报, 2019, 41(5): 93-102.
GAO Q H, DONG J C.Feedforward compensation design for observation error of extended state observer[J]. Journal of National University of Defense Technology, 2019, 41(5): 93-102.
[10] ZHENG M H, CHEN X, TOMIZUKA M.Extended state observer with phase compensation to estimate and suppress high-frequency disturbances[C]//2016 American Control Conference (ACC). Boston, MA, USA, 2016: 3521-3526.
[11] 杨林, 曾江, 马文杰, 等. 基于改进二阶线性自抗扰技术的微网逆变器电压控制[J]. 电力系统自动化, 2019, 43(4): 146-153.
YANG L, ZENG J, MA W J, et al.Voltage control of microgrid inverter based on improved second-order linear active disturbance rejection control[J]. Automation of electric power systems, 2019, 43(4): 146-153.
[12] 马幼捷, 袁业沧, 周雪松, 等. 模型信息联合校正型自抗扰控制策略[J]. 太阳能学报, 2024, 45(3): 389-398.
MA Y J, YUAN Y C, ZHOU X S, et al.Model information combined correction active disturbance rejection voltage stabilizing control strategy[J]. Acta energiae solaris sinica, 2024, 45(3): 389-398.
[13] 王海强, 黄海. 扩张状态观测器的性能与应用[J]. 控制与决策, 2013, 28(7): 1078-1082.
WANG H Q, HUANG H.Property and applications of extended state observer[J]. Control and decision, 2013, 28(7): 1078-1082.
[14] 刘玉燕, 刘吉臻, 周世梁. 基于降阶状态观测器的压水堆功率自抗扰控制[J]. 中国电机工程学报, 2017, 37(22): 6666-6674.
LIU Y Y, LIU J Z, ZHOU S L.Active disturbance rejection control of pressurized water reactor power based on reduced-order extended state observer[J]. Proceedings of the CSEE, 2017, 37(22): 6666-6674.
[15] 王军晓, 戎佳艺, 俞立. 直流降压变换器的降阶扩张状态观测器与滑模控制设计与实现[J]. 控制理论与应用, 2019, 36(9): 1486-1492.
WANG J X, RONG J Y, YU L.Design and implementation of reduced-order extended state observer and sliding mode control for DC-DC buck converter[J]. Control theory & applications, 2019, 36(9): 1486-1492.
[16] 周兰, 姜福喜, 潘昌忠, 等. 基于降阶扩张状态观测器的重复控制系统设计[J]. 控制与决策, 2022, 37(4): 933-943.
ZHOU L, JIANG F X, PAN C Z, et al.A method of designing a reduced-order-extended-observer-based repetitive-control system[J]. Control and decision, 2022, 37(4): 933-943.
[17] 卜飞飞, 郭子韬, 顾毅君, 等. 基于改进型降阶观测器的永磁直驱伺服电动机转矩扰动抑制策略[J]. 电工技术学报, 2022, 37(16): 4104-4115.
BU F F, GUO Z T, GU Y J, et al.Torque disturbance suppression strategy of permanent magnet direct drive servo motor based on improved reduced order observer[J]. Transactions of China Electrotechnical Society, 2022, 37(16): 4104-4115.
[18] WEI W, ZHANG Z Y, ZUO M.Phase leading active disturbance rejection control for a nanopositioning stage[J]. ISA transactions, 2021, 116: 218-231.
[19] GHEISARNEJAD M, FARSIZADEH H, KHOOBAN M H.A novel nonlinear deep reinforcement learning controller for DC-DC power buck converters[J]. IEEE transactions on industrial electronics, 2021, 68(8): 6849-6858.
[20] 周雪松, 张心茹, 赵浛宇, 等. 基于DDPG算法的微网负载端接口变换器自抗扰控制[J]. 电力系统保护与控制, 2023, 51(21): 66-75.
ZHOU X S, ZHANG X R, ZHAO H Y, et al.Active disturbance rejection control of a microgrid load-side interface converter based on a DDPG algorithm[J]. Power system protection and control, 2023, 51(21): 66-75.
[21] 袁东, 马晓军, 曾庆含, 等. 二阶系统线性自抗扰控制器频带特性与参数配置研究[J]. 控制理论与应用, 2013, 30(12): 1630-1640.
YUAN D, MA X J, ZENG Q H, et al.Research on frequency-band characteristics and parameters configuration of linear active disturbance rejection control for second-order systems[J]. Control theory & applications, 2013, 30(12): 1630-1640.