深度强化学习优化非线性自抗扰微网稳压技术

周雪松; 刘曜荣; 马幼捷; 陶珑; 王馨悦; 问虎龙

doi:10.19912/j.0254-0096.tynxb.2025-0003

PDF(1414 KB)

太阳能学报 ›› 2026, Vol. 47 ›› Issue (6) : 1-9. DOI: 10.19912/j.0254-0096.tynxb.2025-0003

深度强化学习优化非线性自抗扰微网稳压技术

周雪松¹, 刘曜荣¹, 马幼捷¹, 陶珑¹, 王馨悦¹, 问虎龙^2,3

作者信息 +

DEEP REINFORCEMENT LEARNING OPTIMIZATION OF NONLINEAR ACTIVE DISTURBANCE REJECTION CONTROL FOR MICROGRID VOLTAGE STABILIZATION

Zhou Xuesong¹, Liu Yaorong¹, Ma Youjie¹, Tao Long¹, Wang Xinyue¹, Wen Hulong^2,3

Author information +

文章历史 +

摘要

针对新能源分布式利用中风光储微网电压稳定性较差的问题,提出一种基于SAC算法优化非线性自抗扰控制策略（SAC-ADRC）。首先,建立非线性ADRC控制的风光储系统模型;然后,通过利用线性/非线性自抗扰控制切换的方法重构非线性ADRC的增益参数,改善其内部参数较难整定和分析的问题;最后,分析建立SAC智能体与微网环境交互学习的机制,实现对非线性ADRC参数的调整。利用算法收敛曲线和模拟各种经典工况进行对比分析,验证SAC-ADRC控制策略在抗扰性能上所带来的优越性。由此表明,非线性ADRC和深度强化学习的有机融合,对提高微网母线电压稳定性具有良好效果。

Abstract

To address the problem of poor voltage stability of wind-solar-energy storage DC microgrids in distributed renewable energy systems, an optimized nonlinear active disturbance rejection control strategy based on the SAC algorithm （SAC-ADRC） is proposed. Firstly, the wind-solar-energy storage system is modeled with nonlinear ADRC control. Then, the gain parameters of the nonlinear ADRC are reconstructed by using linear/nonlinear ADRC switching to improve its internal parameters which are more difficult to tune and analyze. Finally, the analysis establishes a mechanism for SAC intelligence to learn interactively with the microgrid environment, enabling the adjustment of non-linear ADRC parameters. Comparative analysis using algorithm convergence curves and simulation of various classical working conditions confirms the superiority of the SAC-ADRC control strategy in terms of interference performance. Thus, it is shown that the organic integration of nonlinear ADRC and deep reinforcement learning improves the stability of the microgrid bus voltage.

导出引用

周雪松, 刘曜荣, 马幼捷, 陶珑, 王馨悦, 问虎龙. 深度强化学习优化非线性自抗扰微网稳压技术[J]. 太阳能学报. 2026, 47(6): 1-9 https://doi.org/10.19912/j.0254-0096.tynxb.2025-0003

Zhou Xuesong, Liu Yaorong, Ma Youjie, Tao Long, Wang Xinyue, Wen Hulong. DEEP REINFORCEMENT LEARNING OPTIMIZATION OF NONLINEAR ACTIVE DISTURBANCE REJECTION CONTROL FOR MICROGRID VOLTAGE STABILIZATION[J]. Acta Energiae Solaris Sinica. 2026, 47(6): 1-9 https://doi.org/10.19912/j.0254-0096.tynxb.2025-0003

中图分类号： TM46 TP273

参考文献

[1] 王继业. 人工智能赋能源网荷储协同互动的应用及展望[J]. 中国电机工程学报, 2022, 42(21): 7667-7682.
WANG J Y.Application and prospect of source-grid-load-storage coordination enabled by artificial intelligence[J]. Proceedings of the CSEE, 2022, 42(21): 7667-7682.
[2] 王雨祺, 赵兴勇, 赖建中, 等. 基于自抗扰控制的光伏微电网储能控制策略[J]. 电气工程学报, 2024, 19(3): 423-431.
WANG Y Q, ZHAO X Y, LAI J Z, et al.Energy storage control strategy for photovoltaic microgrid based on active disturbance rejection control[J]. Journal of electrical engineering, 2024, 19(3): 423-431.
[3] 陶珑, 马小勇, 王议锋, 等. 混合储能系统增强型自抗扰协调控制[J]. 太阳能学报, 2024, 45(9): 668-677.
TAO L, MA X Y, WANG Y F, et al.Coordinated control of hybrid energy storage based on enhanced active disturbance rejection control[J]. Acta energiae solaris sinica, 2024, 45(9): 668-677.
[4] 尚彤, 崔学深, 徐明荣, 等. 蓄电池-超级电容混合储能系统放电控制策略[J]. 电源技术, 2017, 41(4): 595-597,637.
SHANG T, CUI X S, XU M R, et al.Discharge control strategies of supercapacitor-battery hybrid energy storage system[J]. Chinese journal of power sources, 2017, 41(4): 595-597,637.
[5] 单锁兰, 谷志锋, 李宏城, 等. 混合储能系统分散自适应强扭曲虚拟阻抗控制[J]. 中国测试, 2021, 47(1): 127-132.
SHAN S L, GU Z F, LI H C, et al.Decentralized adaptive super-twisting virtual impedance control for hybrid energy storage system[J]. China measurement & test, 2021, 47(1): 127-132.
[6] 郑昊, 杜贵平, 雷雁雄, 等. 混合储能双向DC/DC变换器线性自抗扰控制研究[J]. 电力电子技术, 2022, 56(9): 80-83.
ZHENG H, DU G P, LEI Y X, et al.Research on linear active disturbance rejection control of hybrid energy storage bidirectional DC/DC converter[J]. Power electronics, 2022, 56(9): 80-83.
[7] 刘恋, 郑植. 基于改进贪心算法的分布式光伏配网储能系统容量优化配置[J]. 电工技术, 2024(17): 50-52,55.
LIU L, ZHENG Z.Modified greed algorithm-based optimized capacity allocation of energy storage for distributed photovoltaics-integrated distribution networks[J]. Electric engineering, 2024(17): 50-52,55.
[8] 马幼捷, 杨清, 周雪松, 等. 基于模糊神经网络的储能侧双向DC-DC变换器自抗扰控制策略[J]. 太阳能学报, 2023, 44(10): 488-495.
MA Y J, YANG Q, ZHOU X S, et al.Active disturbance rejection control strategy of bidirectional DC-DC converter on energy storage side based on fuzzy neural network[J]. Acta energiae solaris sinica, 2023, 44(10): 488-495.
[9] 龚春阳, 林嘉伟, 黄冬梅, 等. 储能系统双向Buck-Boost变换器控制策略研究[J]. 太阳能学报, 2023, 44(2): 229-238.
GONG C Y, LIN J W, HUANG D M, et al.Research on control strategy of bidirectional Buck-Boost converter in energy storage system[J]. Acta energiae solaris sinica, 2023, 44(2): 229-238.
[10] 杨丰萍, 程权, 周铭栀, 等. 基于改进自抗扰控制的微电网混合储能控制策略[J]. 科学技术与工程, 2022, 22(5): 1898-1905.
YANG F P, CHENG Q, ZHOU M Z, et al.Microgrid hybrid energy storage control strategy based on improved active disturbance rejection control[J]. Science technology and engineering, 2022, 22(5): 1898-1905.
[11] 杨惠, 骆姗, 孙向东, 等. 光伏储能双向DC-DC变换器的自抗扰控制方法研究[J]. 太阳能学报, 2018, 39(5): 1342-1350.
YANG H, LUO S, SUN X D, et al.Research on ADRC method for bidirectional DC-DC converter of solar energy storage system[J]. Acta energiae solaris sinica, 2018, 39(5): 1342-1350.
[12] 刘凯, 杨宝庆. 一种基于自抗扰方法的姿控直气复合控制系统设计[J]. 空天防御, 2021, 4(2): 40-47.
LIU K, YANG B Q.Compound control system design with lateral thrust and aerodynamic force based on ADRC method[J]. Air & space defense, 2021, 4(2): 40-47.
[13] 李杰, 齐晓慧, 夏元清, 等. 线性/非线性自抗扰切换控制方法研究[J]. 自动化学报, 2016, 42(2): 202-212.
LI J, QI X H, XIA Y Q, et al.On linear/nonlinear active disturbance rejection switching control[J]. Acta automatica sinica, 2016, 42(2): 202-212.
[14] 贺庆辰, 秦斌. 基于改进SAC算法的城轨列车混合储能系统动态功率分配策略[J]. 湖南电力, 2024, 44(4): 11-19.
HE Q C, QIN B.Dynamic power allocation strategy for hybrid energy storage system of urban rail trains based on improved SAC algorithm[J]. Hunan electric power, 2024, 44(4): 11-19.
[15] 张忠铝. 基于SAC算法的AUV路径规划与跟踪方法研究[D]. 济南: 山东大学, 2023.
ZHANG Z L.Research on AUV path planning and tracking methods based on SAC algorithm[D]. Ji’nan: Shandong University, 2023.
[16] 苑守正, 刘志林, 郑林熇, 等. 基于事件触发自适应时域MPC的船舶靠泊方法[J]. 控制与决策, 2024, 39(1): 336-344.
YUAN S Z, LIU Z L, ZHENG L H, et al.Ship berthing based on event-triggered adaptive horizon MPC[J]. Control and decision, 2024, 39(1): 336-344.