针对新型电力系统在面临新能源的接入、负载时空分布等多重不确定性的作用下的输出扰动问题,提出一种基于双延迟深度确定性策略梯度（TD3）提出一种基于双延迟深度确定性策略梯度（TD3）算法的重构型线性自抗扰控制策略(TD3-R_LADRC),旨在提升储能系统对直流母线电压波动的平抑能力。首先,针对扩张状态观测器的扰动微分进行观测并进行降阶处理,实现对系统中总扰动因素的更快速、更精准的跟踪与补偿,有效提升控制器的响应速度与准确性。随后,对所提出的改进策略进行频域性能与稳定性分析。为进一步挖掘该策略的性能潜力,引入TD3强化学习算法,对改进后的线性自抗扰控制器的观测器带宽与控制器带宽等关键参数进行智能优化训练。最后,通过数字仿真实验与小功率实验,对比分析在不同工况下改进后控制器与传统线性自抗扰控制以及双闭环PI控制策略在抗扰性、稳定性以及鲁棒性方面的表现。结果表明,所提TD3-R_LADRC策略在面对新能源出力不确定性、负荷波动及外部扰动时表现出更好的控制性能,能够有效增强系统鲁棒性,改善频率稳定控制效果,具有一定的理论意义和工程应用价值。
Abstract
To address the output disturbance issue of new power systems under multiple uncertainties, such as the integration of new energy sources and the spatiotemporal distribution of loads, this paper proposes A refactoring linear active disturbance rejection control strategy (TD3-R_LADRC) based on the twin delayed deep deterministic policy gradient (TD3) algorithm. The objective is to enhance the ability of energy storage systems to mitigate DC bus voltage fluctuations. Firstly, the disturbance differential in the extended state observer （ESO） is observed, and the ESO is reduced in order This enables faster and more precise tracking and compensation of total disturbance factors in the system, effectively improving the response speed and accuracy of controllers. Subsequently, frequency domain performance and stability analysis are conducted on the proposed improved strategy. To further tap into the performance potential of this strategy, this study introduces the TD3 reinforcement learning algorithm and performs intelligent optimization training on key parameters of the improved LADRC, including observer bandwidth and controller bandwidth. Finally, through digital simulation and low-power experiments, this study conducts a comparative analysis of the improved controller, traditional linear active disturbance rejection control, and the dual closed-loop PI control strategy in terms of disturbance rejection, stability, and robustness under different operating conditions. The results demonstrate that the proposed TD3-R_LADRC strategy exhibits better control performance when facing the uncertainty of new energy output, load fluctuations and external disturbances. It can effectively enhance system robustness, improve the control effect of frequency stability, and has certain theoretical significance and engineering application value.
关键词
储能 /
微电网 /
DC-DC变换器 /
频域分析 /
深度强化学习 /
线性自抗扰控制
Key words
energy storage /
microgrid /
DC-DC converters /
frequency domain analysis /
deep reinforcement learning /
linear active disturbance rejection control
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 周喜. 新能源发电在电力系统中的应用研究[J]. 电气技术与经济, 2023(6): 180-182.
Zhou X.Research on application of new energy power generation in power system[J]. Electrical Equipment and Economy, 2023(6): 180-182.
[2] 肖传奇, 王紫塔, 叶赓, 等. 新型电力系统背景下电网发展业务数字化转型架构及路径分析[J]. 电器工业, 2023(9): 75-78.
Xiao C Q, Wang Z T, Ye G, et al.Digital transformation framework and path analysis of power grid development business under the background of new power system[J]. China Electrical Equipment Industry, 2023(9): 75-78.
[3] 唐西胜, 李伟, 沈晓东. 面向新型电力系统的储能规划方法研究进展及展望[J]. 电力系统自动化, 2024, 48(9): 178-191.
Tang X S, Li W, Shen X D.Research progress and prospect of energy storage planning method for new power system[J]. Automation of Electric Power Systems, 2024, 48(9): 178-191.
[4] 居文平, 王一帆, 赵勇, 等. 新型电力系统长时储能技术综述[J]. 热力发电, 2024, 53(9): 1-9.
Ju W P, Wang Y F, Zhao Y, et al.Overview of long-term energy storage technologies in new power systems[J]. Thermal Power Generation, 2024, 53(9): 1-9.
[5] 马幼捷, 庞笑琴, 周雪松, 等. 光伏储能侧接口变换器自适应协调自抗扰控制[J]. 太阳能学报, 2024, 45(11): 636-646.
Ma Y J, Pang X Q, Zhou X S, et al.Adaptive and coordinated active disturbance rejection control of interface converter at photovoltaic energy storage side[J]. Acta Energiae Solaris Sinica, 2024, 45(11): 636-646.
[6] 张伟亮, 张辉, 支娜, 等. 基于节点源荷电流差分的直流微电网储能变换器控制策略[J]. 电工技术学报, 2022, 37(9): 2199-2210.
Zhang W L, Zhang H, Zhi N, et al.Control strategy of DC microgrid energy storage converter based on node differential current[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2199-2210.
[7] 吴忠, 刘朝辉. 基于电流模式的DC/DC升压变换器非线性PI控制[J]. 中国电机工程学报, 2011, 31(33): 31-36.
Wu Z, Liu C H.Nonlinear PI control of DC/DC boost power converters based on current mode[J]. Proceedings of the CSEE, 2011, 31(33): 31-36.
[8] 李虹, 尚佳宁, 陈姚, 等. 基于fal函数的非线性PI控制器在DC-DC变换器中的应用[J]. 电工技术学报, 2014, 29(S1): 326-331.
Li H, Shang J N, Chen Y, et al.The applications of nonlinear PI controller based on the fal function in the DC-DC converter[J]. Transactions of China Electrotechnical Society, 2014, 29(S1): 326-331.
[9] Han J Q.From PID to active disturbance rejection control[J]. IEEE Transactions on Industrial Electronics, 2009, 56(3): 900-906.
[10] 李乔, 吴捷. 自抗扰控制及其在DC-DC变换器中的应用[J]. 电工技术学报, 2005, 20(1): 83-88.
Li Q, Wu J.Auto-disturbance rejection control and its application to DC-DC converter[J]. Transactions of China Electrotechnical Society, 2005, 20(1): 83-88.
[11] Gao Z Q.Scaling and bandwidth-parameterization based controller tuning[C]//Proceedings of the 2003 American Control Conference, 2003. Denver, CO, USA, 2003: 4989-4996
[12] 王晓雷, 李郎, 郭飞亚, 等. 全桥LLC谐振变换器的线性自抗扰控制[J]. 机械工程与自动化, 2024, 53(6): 20-23.
Wang X L, Li L, Guo F Y, et al.Linear active disturbance rejection control of a full-bridge LLC resonant converter[J]. Mechanical Engineering & Automation, 2024, 53(6): 20-23.
[13] 周雪松, 王馨悦, 马幼捷, 等. 直流微网变增益专家自抗扰控制[J]. 电力系统保护与控制, 2023, 51(18): 70-80.
Zhou X S, Wang X Y, Ma Y J, et al.Expert system-changeable gain ADRC for a DC microgrid[J]. Power System Protection and Control, 2023, 51(18): 70-80.
[14] 陈超波, 王霞霞, 高嵩, 等. 基于区间滑模观测器的逆变器开路故障诊断方法[J]. 中国电机工程学报, 2020, 40(14): 4569-4579.
Chen C B, Wang X X, Gao S, et al.A diagnosis method for open-circuit faults in inverters based on interval sliding mode observer[J]. Proceedings of the CSEE, 2020, 40(14): 4569-4579.
[15] 孙斌, 王海霞, 苏涛, 等. 永磁同步电机调速系统非线性自抗扰控制器设计与参数整定[J]. 中国电机工程学报, 2020, 40(20): 6715-6725.
Sun B, Wang H X, Su T, et al.Nonlinear active disturbance rejection controller design and tuning for permanent magnet synchronous motor speed control system[J]. Proceedings of the CSEE, 2020, 40(20): 6715-6725.
[16] 周雪松, 周泳良, 马幼捷, 等. 风电并网逆变器的改进型线性自抗扰控制[J]. 电气传动, 2022, 52(1): 46-53.
Zhou X S, Zhou Y L, Ma Y J, et al.Improved linear active disturbance rejection control of wind power grid-connected inverter[J]. Electric Drive, 2022, 52(1): 46-53.
[17] 周雪松, 张心茹, 赵浛宇, 等. 基于DDPG算法的微网负载端接口变换器自抗扰控制[J]. 电力系统保护与控制, 2023, 51(21): 66-75.
Zhou X S, Zhang X R, Zhao H Y, et al.Active disturbance rejection control of a microgrid load-side interface converter based on a DDPG algorithm[J]. Power System Protection and Control, 2023, 51(21): 66-75.
[18] 周国华, 冷敏瑞, 李媛, 等. 开关变换器及其控制环路的建模综述[J]. 中国电机工程学报, 2020, 40(1): 183-199.
Zhou G H, Leng M R, Li Y, et al.A review on modeling of switching converters and their control loops[J]. Proceedings of the Chinese Society for Electrical Engineering, 2020, 40(1): 183-199.
[19] 龚春阳, 林嘉伟, 黄冬梅, 等. 储能系统双向Buck-Boost变换器控制策略研究[J]. 太阳能学报, 2023, 44(2): 229-238.
Gong C Y, Lin J W, Huang D M, et al.Research on control strategy of bidirectional buck-boost converter in energy storage system[J]. Acta Energiae Solaris Sinica, 2023, 44(2): 229-238.
[20] 高志强. 自抗扰控制思想探究[J]. 控制理论与应用, 2013, 30(12): 1498-1510.
Gao Z Q.On the foundation of active disturbance rejection control[J]. Control Theory & Applications, 2013, 30(12): 1498-1510.
[21] 袁东,马晓军,曾庆含,等.二阶系统线性自抗扰控制器频带特性与参数配置研究[J]. 控制理论与应用, 2013, 30(12): 1630-1640.
Yuan D, Ma X J, Zeng Q H, et al.Research on frequency-band characteristics and parameters configuration of linear active disturbance rejection control for second-order sys-tems[J]. Control Theory & Applications, 2013, 30(12): 1630-1640.
[22] Chen S, Xue W C, Zhong S, et al.On comparison of modified ADRCs for nonlinear uncertain systems with time delay[J]. Science China Information Sciences, 2018, 61(7): 70223.
[23] 郝禹哲, 王振雷. 安全约束下合作型多智能体TD3算法[J]. 计算机应用研究, 2023, 40(6): 1692-1696, 1701.
Hao Y Z, Wang Z L.Cooperative multi-agent TD3 algorithm under security constraints[J]. Application Research of Computers, 2023, 40(6): 1692-1696, 1701.
[24] Fan Y X, Dong H Y, Zhao X W, et al.Path-following control of unmanned underwater vehicle based on an improved TD3 deep reinforcement learning[J]. IEEE Transactions on Control Systems Technology, 2024, 32(5): 1904-1919.
[25] 皇金锋, 杨振宇, 李帅杰. 基于混合储能的光储直流微网改进型滑模自抗扰控制[J]. 电力系统保护与控制, 2023, 51(23): 151-159.
Huang J F, Yang Z Y, Li S J.Improved sliding mode self-disturbance resistance control of an optical storage DC micro-grid based on hybrid energy storage[J]. Power System Protection and Control, 2023, 51(23): 151-159.
基金
国家自然科学基金（U24B6011）; 国家自然科学基金重点项目（U23B20142）
PDF(2526 KB)
