CONDITIONAL ENTROPY BASED SAC ALGORITHM FOR ENERGY STORAGE CONVERTER ACTIVE DISTURBANCE REJECTION CONTROL VOLTAGE STABILISATION STRATEGY

Ma Youjie; Han Dexin; Zhou Xuesong; Zhang Yuxuan; Wang Xinyue; Tao Long

doi:10.19912/j.0254-0096.tynxb.2024-2277

PDF(5337 KB)

Acta Energiae Solaris Sinica ›› 2026, Vol. 47 ›› Issue (5) : 457-467. DOI: 10.19912/j.0254-0096.tynxb.2024-2277

CONDITIONAL ENTROPY BASED SAC ALGORITHM FOR ENERGY STORAGE CONVERTER ACTIVE DISTURBANCE REJECTION CONTROL VOLTAGE STABILISATION STRATEGY

Ma Youjie, Han Dexin, Zhou Xuesong, Zhang Yuxuan, Wang Xinyue, Tao Long

Author information +

History +

Abstract

Aiming at the problem that DC bus voltage is susceptible to PV output power changes and load disturbances, an Active Disturbance Rejection Control strategy based on conditional entropy and the deep reinforcement learning Soft Actor-Critic （SAC） algorithm is designed. By introducing the conditional entropy as an evaluation index to measure the degree of perturbation of the system and combining it with the deep reinforcement learning algorithm, real-time self-driven optimal tuning of the controller bandwidth is carried out to ensure that the energy storage unit containing a DC/DC converter can suppress the bus voltage fluctuation and improve the safety and stability performance of the DC microgrid. Finally, the feasibility of the proposed strategy is demonstrated by comparing it with the double-closed-loop PI control and the traditional linear active disturbance rejection control under different operating conditions.

Key words

DC microgrid / DC/DC converter / active disturbance rejection control / SAC / conditional entropy

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Ma Youjie, Han Dexin, Zhou Xuesong, Zhang Yuxuan, Wang Xinyue, Tao Long. CONDITIONAL ENTROPY BASED SAC ALGORITHM FOR ENERGY STORAGE CONVERTER ACTIVE DISTURBANCE REJECTION CONTROL VOLTAGE STABILISATION STRATEGY[J]. Acta Energiae Solaris Sinica. 2026, 47(5): 457-467 https://doi.org/10.19912/j.0254-0096.tynxb.2024-2277

References

[1] 张明霞, 闫涛, 来小康, 等. 电网新功能形态下储能技术的发展愿景和技术路径[J]. 电网技术, 2018, 42(5): 1370-1377.
ZHANG M X, YAN T, LAI X K, et al.Technology vision and route of energy storage under new power grid function configuration[J]. Power system technology, 2018, 42(5): 1370-1377.
[2] 曾浩, 赵恩盛, 周思宇, 等. 基于电流一致性的直流微网自适应下垂控制[J]. 电力系统保护与控制, 2022, 50(12): 11-21.
ZENG H, ZHAO E S, ZHOU S Y, et al.Adaptive droop control of a DC microgrid based on current consistency[J]. Power system protection and control, 2022, 50(12): 11-21.
[3] 马幼捷, 杨清, 周雪松, 等. 基于模糊神经网络的储能侧双向DC-DC变换器自抗扰控制策略[J]. 太阳能学报, 2023, 44(10): 488-495.
MA Y J, YANG Q, ZHOU X S, et al.Active disturbance rejection control strategy of bidirectional DC-DC converter on energy storage side based on fuzzy neural network[J]. Acta energiae solaris sinica, 2023, 44(10): 488-495.
[4] 马帅旗, 任思嘉, 贺海育, 等. 储能交错并联双向DC-DC变换器的自抗扰控制[J]. 电子测量技术, 2024, 47(13): 35-44.
MA S Q, REN S J, HE H Y, et al.Active disturbance rejection control of energy storage interleaved parallel bidirectional DC-DC converter[J]. Electronic measurement technology, 2024, 47(13): 35-44.
[5] 刘晓, 刘立群, 吴祥辉, 等. 双向DC/DC变换器在直流微网中的应用[J]. 太原科技大学学报, 2016, 37(2): 108-113.
LIU X, LIU L Q, WU X H, et al.The application of bi-directional DC/DC converter in micro DC-grid[J]. Journal of Taiyuan University of Science and Technology, 2016, 37(2): 108-113.
[6] 杨惠, 骆姗, 孙向东, 等. 光伏储能双向DC-DC变换器的自抗扰控制方法研究[J]. 太阳能学报, 2018, 39(5): 1342-1350.
YANG H, LUO S, SUN X D, et al.Research on ADRC method for bidirectional DC-DC converter of solar energy storage system[J]. Acta energiae solaris sinica, 2018, 39(5): 1342-1350.
[7] 李浩东, 肖伸平, 余锦. 基于自适应PSO的微电网双向DC-DC变换器前馈自抗扰控制[J]. 电机与控制应用, 2023, 50(2): 73-81.
LI H D, XIAO S P, YU J.Feedforward active disturbance rejection control of microgrid bidirectional DC-DC converter based on adaptive PSO[J]. Electric machines & control application, 2023, 50(2): 73-81.
[8] 禹梅, 杨倩. 基于遗传算法整定参数的LADRC在LFC时滞问题中的应用[C]//第25届中国控制与决策会议论文集, 中国贵阳, 2013: 6.
YU M, YANG Q.LFC design with time-delay via LADRC and setting parameters based on genetic algorithm[C]//25th Chinese Control and Decision Conference (CCDC). Guiyang, China, 2013: 6.
[9] 周雪松, 杨子明, 马幼捷. 基于神经网络的最优带宽风电并网自抗扰控制[J]. 太阳能学报, 2022, 43(9): 226-235.
ZHOU X S, YANG Z M, MA Y J.Optimal bandwidth active disturbance rejection control for wind turbine grid-connection based on neural network[J]. Acta energiae solaris sinica, 2022, 43(9): 226-235.
[10] 缪刘洋, 朱其新, 朱永红. 基于多智能体强化学习的滑模控制器参数整定[J]. 机床与液压, 2024, 52(11): 160-166.
MIAO L Y, ZHU Q X, ZHU Y H.Parameter tuning of sliding mode controller based on reinforcement learning of multiple intelligences[J]. Machine tool & hydraulics, 2024, 52(11): 160-166.
[11] 周雪松, 谢冰杰, 马幼捷, 等. 基于改进自抗扰的双向DC-DC变换器控制策略研究[J]. 电气传动, 2022, 52(15): 10-16, 30.
ZHOU X S, XIE B J, MA Y J, et al.Research on control strategy of bidirectional DC-DC converter based on improved active disturbance rejection[J]. Electric drive, 2022, 52(15): 10-16, 30.
[12] 王瑶. 基于状态空间平均模型的电压控制SIDO Buck变换器稳定性分析[J]. 中国电机工程学报, 2018, 38(6): 1810-1817, 1918.
WANG Y.Stability analysis for voltage controlled single-inductor dual-output Buck converter based on state space average model[J]. Proceedings of the CSEE, 2018, 38(6): 1810-1817, 1918.
[13] 周雪松, 王馨悦, 马幼捷, 等. 直流微网变增益专家自抗扰控制[J]. 电力系统保护与控制, 2023, 51(18): 70-80.
ZHOU X S, WANG X Y, MA Y J, et al.Expert system-changeable gain ADRC for a DC microgrid[J]. Power system protection and control, 2023, 51(18): 70-80.
[14] 李健, 谭文, 张彬文. 观测器带宽参数化的自抗扰控制[J]. 控制工程, 2022, 29(7): 1181-1186.
LI J, TAN W, ZHANG B W.Observer-bandwidth-based active disturbance rejection control[J]. Control engineering of China, 2022, 29(7): 1181-1186
[15] 傅友, 邹东升. SDN中基于条件熵和决策树的DDoS攻击检测方法[J]. 重庆大学学报, 2023, 46(7): 1-8.
FU Y, ZOU D S.A DDoS attack detection method based on conditional entropy and decision tree in SDN[J]. Journal of Chongqing University, 2023, 46(7): 1-8.
[16] 周雪松, 张心茹, 赵浛宇, 等. 基于DDPG算法的微网负载端接口变换器自抗扰控制[J]. 电力系统保护与控制, 2023, 51(21): 66-75.
ZHOU X S, ZHANG X R, ZHAO H Y, et al.Active disturbance rejection control of a microgrid load-side interface converter based on a DDPG algorithm[J]. Power system protection and control, 2023, 51(21): 66-75.
[17] 夏琳, 罗威, 王俊霞, 等. 基于后验经验回放的MAAC多智能体强化学习算法[J]. 软件, 2023, 44(2): 17-22, 41.
XIA L, LUO W, WANG J X, et al.Hindsight experience replay machanism of MAAC algorithm[J]. Software, 2023, 44(2): 17-22, 41.
[18] 鄢霞, 何勇, 张庆铭, 等. 基于SAC的永磁同步电机智能控制算法[J]. 组合机床与自动化加工技术, 2023(9): 86-91.
YAN X, HE Y, ZHANG Q M, et al.Intelligent control algorithm of permanent magnet synchronous motor based on SAC[J]. Modular machine tool & automatic manufacturing technique, 2023(9): 86-91.
[19] HAARNOJA T, ZHOU A, HARTIKAINEN K, et al. Soft actor-critic algorithms and applications[EB/OL].2018: arXiv: 1812.05905. https://arxiv.org/abs/1812.05905
[20] HAARNOJA T, ZHOU A, ABBEEL P, et al.Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor.[C]//International conference on machine learning. Pmlr, 2018: 1861-1870.
[21] 陶力, 杨夏喜, 顾金辉, 等. 基于SAC和TD3的含电动汽车虚拟电厂调度策略[J]. 电气传动, 2023, 53(9): 25-34.
TAO L, YANG X X, GU J H, et al.Scheduling strategy of virtual power plant with electric vehicle based on SAC and TD3[J]. Electric drive, 2023, 53(9): 25-34.
[22] 张晓莉, 郭仕林, 刘鼎, 等. 基于改进SAC的倒立摆控制算法研究[J]. 电子测量技术, 2024, 47(1): 93-100.
ZHANG X L, GUO S L, LIU D, et al.Research on the control algorithm of inverted pendulum based on improved SAC[J]. Electronic measurement technology, 2024, 47(1): 93-100.
[23] 马彬, 石永乐, 柴浩男, 等. 基于时变权重模型预测的双向DC-DC优化控制方法[J]. 重庆理工大学学报(自然科学), 2024, 38(7): 12-20.
MA B, SHI Y L, CHAI H N, et al.Bidirectional DC-DC optimal control method based on time-varying weight model prediction[J]. Journal of Chongqing University of Technology (natural science), 2024, 38(7): 12-20.