针对光储直流微电网母线电压控制过程存在的相位滞后问题,以储能双向DC-DC变换器为研究对象,提出一种含相位补偿网络的深度强化学习自抗扰控制策略。在该策略中,首先,将线性扩张状态观测器降阶处理,在其总和扰动作用通道串联相位补偿网络,以提供扰动估计所需的超前相角;其次,分析补偿环节改善扰动抑制性能的机理,配置补偿参数取值区间,评估改进后观测器对于典型时变扰动信号的跟踪特性;然后,基于深度强化学习算法建立控制变量优化模型,通过训练智能体与环境不断交互,探索变换器最优策略参数空间,实现观测器带宽和校正系数的自适应调整;最后,仿真对比典型工况下不同控制方式对于母线电压扰动的跟踪能力和控制精度。结果表明,所提控制策略有效可行。
Abstract
To address the phase lag issue in bus voltage control of DC microgrids, a deep reinforcement learning-based active disturbance rejection control strategy incorporating a phase compensation network is developed for bidirectional DC-DC converters in energy storage systems. In this strategy,the order of the linear extended state observer is reduced, and a phase compensation network is connected in series with its total disturbance channel to provide the required lead phase angle. Subsequently,the mechanism by which the compensation link enhances disturbance suppression performance is analyzed,and the compensation parameter ranges are configured. Besides,a control parameter optimization model is established based on deep reinforcement learning algorithms. By training an intelligent agent to interact with the DC microgrid environment,the optimal combination of policy parameters is explored to achieve adaptive adjustment of observer bandwidth and correction parameters. Finally,simulations are conducted to compare the tracking ability and control accuracy of different control methods for bus voltage disturbances under typical operating conditions,and the results confirm the effectiveness of the proposed control strategy.
关键词
储能 /
微电网 /
功率变换器 /
深度强化学习 /
马尔科夫过程 /
抗扰
Key words
energy storage /
microgrids /
power converters /
deep reinforcement learning /
Markov processes /
disturbance rejection
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 张伟亮, 张辉, 支娜, 等. 基于节点源荷电流差分的直流微电网储能变换器控制策略[J]. 电工技术学报, 2022, 37(9): 2199-2210.
ZHANG W L, ZHANG H, ZHI N, et al.Control strategy of DC microgrid energy storage converter based on node differential current[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2199-2210.
[2] 薛花, 胡英俊, 董丙伟, 等. 基于分层控制的光-储-燃直流供电系统能量管理方法[J]. 电力系统自动化, 2018, 42(21): 53-62.
XUE H, HU Y J, DONG B W, et al.Hierarchical control based energy management method for photovoltaic-energy storage-fuel cell DC generation system[J]. Automation of electric power systems, 2018, 42(21): 53-62.
[3] 代丽, 王君瑞, 谭露, 等. 基于自适应下垂控制的光储直流微网控制策略研究[J]. 太阳能学报, 2024, 45(8): 154-163.
DAI L, WANG J R, TAN L, et al.Research on control strategy of PV storage DC microgrid based on adaptive droop control[J]. Acta energiae solaris sinica, 2024, 45(8): 154-163.
[4] 曾国辉, 朱相臣, 曾志伟, 等. 具有公共低压直流母线电压支撑功能的储能单元SOC自动均衡控制策略[J]. 中国电机工程学报, 2022, 42(19): 7160-7170.
ZENG G H, ZHU X C, ZENG Z W, et al.Energy storage unit state-of-charge automatic equalization control strategy with common LVDC bus voltage support function[J]. Proceedings of the CSEE, 2022, 42(19): 7160-7170.
[5] THOMAS T, MISHRA M K, KUMAR C, et al.Control of a PV-wind based DC microgrid with hybrid energy storage system using Lyapunov approach and sliding mode control[J]. IEEE transactions on industry applications, 2024, 60(2): 3746-3758.
[6] 马幼捷, 杨清, 周雪松, 等. 基于模糊神经网络的储能侧双向DC-DC变换器自抗扰控制策略[J]. 太阳能学报, 2023, 44(10): 488-495.
MA Y J, YANG Q, ZHOU X S, et al.Active disturbance rejection control strategy of bidirectional DC-DC converter on energy storage side based on fuzzy neural network[J]. Acta energiae solaris sinica, 2023, 44(10): 488-495.
[7] 张世欣, 皇金锋, 杨艺. 基于平坦理论的直流微电网双向DC-DC变换器改进滑模自抗扰控制[J]. 电力系统保护与控制, 2023, 51(5): 107-116.
ZHANG S X, HUANG J F, YANG Y.Improved sliding mode and active disturbance rejection control based on flatness theory for a bi-directional DC-DC converter in a DC microgrid[J]. Power system protection and control, 2023, 51(5): 107-116.
[8] GAO Z Q.Scaling and bandwidth-parameterization based controller tuning[C]//Proceedings of the 2003 American Control Conference, 2003. Denver, CO, USA, 2003: 4989-4996.
[9] 高钦和, 董家臣. 扩张状态观测器的观测误差前馈补偿设计[J]. 国防科技大学学报, 2019, 41(5): 93-102.
GAO Q H, DONG J C.Feedforward compensation design for observation error of extended state observer[J]. Journal of National University of Defense Technology, 2019, 41(5): 93-102.
[10] ZHENG M H, CHEN X, TOMIZUKA M.Extended state observer with phase compensation to estimate and suppress high-frequency disturbances[C]//2016 American Control Conference (ACC). Boston, MA, USA, 2016: 3521-3526.
[11] 杨林, 曾江, 马文杰, 等. 基于改进二阶线性自抗扰技术的微网逆变器电压控制[J]. 电力系统自动化, 2019, 43(4): 146-153.
YANG L, ZENG J, MA W J, et al.Voltage control of microgrid inverter based on improved second-order linear active disturbance rejection control[J]. Automation of electric power systems, 2019, 43(4): 146-153.
[12] 马幼捷, 袁业沧, 周雪松, 等. 模型信息联合校正型自抗扰控制策略[J]. 太阳能学报, 2024, 45(3): 389-398.
MA Y J, YUAN Y C, ZHOU X S, et al.Model information combined correction active disturbance rejection voltage stabilizing control strategy[J]. Acta energiae solaris sinica, 2024, 45(3): 389-398.
[13] 王海强, 黄海. 扩张状态观测器的性能与应用[J]. 控制与决策, 2013, 28(7): 1078-1082.
WANG H Q, HUANG H.Property and applications of extended state observer[J]. Control and decision, 2013, 28(7): 1078-1082.
[14] 刘玉燕, 刘吉臻, 周世梁. 基于降阶状态观测器的压水堆功率自抗扰控制[J]. 中国电机工程学报, 2017, 37(22): 6666-6674.
LIU Y Y, LIU J Z, ZHOU S L.Active disturbance rejection control of pressurized water reactor power based on reduced-order extended state observer[J]. Proceedings of the CSEE, 2017, 37(22): 6666-6674.
[15] 王军晓, 戎佳艺, 俞立. 直流降压变换器的降阶扩张状态观测器与滑模控制设计与实现[J]. 控制理论与应用, 2019, 36(9): 1486-1492.
WANG J X, RONG J Y, YU L.Design and implementation of reduced-order extended state observer and sliding mode control for DC-DC buck converter[J]. Control theory & applications, 2019, 36(9): 1486-1492.
[16] 周兰, 姜福喜, 潘昌忠, 等. 基于降阶扩张状态观测器的重复控制系统设计[J]. 控制与决策, 2022, 37(4): 933-943.
ZHOU L, JIANG F X, PAN C Z, et al.A method of designing a reduced-order-extended-observer-based repetitive-control system[J]. Control and decision, 2022, 37(4): 933-943.
[17] 卜飞飞, 郭子韬, 顾毅君, 等. 基于改进型降阶观测器的永磁直驱伺服电动机转矩扰动抑制策略[J]. 电工技术学报, 2022, 37(16): 4104-4115.
BU F F, GUO Z T, GU Y J, et al.Torque disturbance suppression strategy of permanent magnet direct drive servo motor based on improved reduced order observer[J]. Transactions of China Electrotechnical Society, 2022, 37(16): 4104-4115.
[18] WEI W, ZHANG Z Y, ZUO M.Phase leading active disturbance rejection control for a nanopositioning stage[J]. ISA transactions, 2021, 116: 218-231.
[19] GHEISARNEJAD M, FARSIZADEH H, KHOOBAN M H.A novel nonlinear deep reinforcement learning controller for DC-DC power buck converters[J]. IEEE transactions on industrial electronics, 2021, 68(8): 6849-6858.
[20] 周雪松, 张心茹, 赵浛宇, 等. 基于DDPG算法的微网负载端接口变换器自抗扰控制[J]. 电力系统保护与控制, 2023, 51(21): 66-75.
ZHOU X S, ZHANG X R, ZHAO H Y, et al.Active disturbance rejection control of a microgrid load-side interface converter based on a DDPG algorithm[J]. Power system protection and control, 2023, 51(21): 66-75.
[21] 袁东, 马晓军, 曾庆含, 等. 二阶系统线性自抗扰控制器频带特性与参数配置研究[J]. 控制理论与应用, 2013, 30(12): 1630-1640.
YUAN D, MA X J, ZENG Q H, et al.Research on frequency-band characteristics and parameters configuration of linear active disturbance rejection control for second-order systems[J]. Control theory & applications, 2013, 30(12): 1630-1640.
基金
国家自然科学基金重点项目（U23B20142）; 天津市研究生科研创新项目（2022BKYZ036）; 安徽省高校自然科学研究项目（KJ2021B09）
PDF(2518 KB)
