ADJUSTMENT PERFORMANCE ANALYSIS OF WIND-HYDRO HYBRID POWER SYSTEM UNDER MINI NEURAL NETWORK REINFORCEMENT LEARNING

Chen Diyi; Dong Wenhui; Yuan Yichen; Xu Beibei

doi:10.19912/j.0254-0096.tynxb.2022-0869

PDF(2348 KB)

Acta Energiae Solaris Sinica ›› 2023, Vol. 44 ›› Issue (10) : 329-338. DOI: 10.19912/j.0254-0096.tynxb.2022-0869

ADJUSTMENT PERFORMANCE ANALYSIS OF WIND-HYDRO HYBRID POWER SYSTEM UNDER MINI NEURAL NETWORK REINFORCEMENT LEARNING

Chen Diyi¹, Dong Wenhui¹, Yuan Yichen¹, Xu Beibei^1,2

Author information +

History +

Abstract

To enhance the electric frequency and power regulation performance under complex wind speed environment, a deep reinforcement learning algorithm based on a mini neural network was applied in the wind-hydro hybrid power system. The proposed algorithm was based on deep deterministic policy gradient, the frequency and power fluctuation amplitude of power grid were took as reward functions, and the control strategy was acquired finally. Further, to improve the training speed, a mini neural network featured by down-sampling and compression deferral was introduced in the algorithm. Finally, the paper compared the traditional PID controller, deep deterministic strategy gradient algorithm and mini neural network reinforcement learning algorithm in a wind-hydro hybrid power system. The results show that the intelligent algorithm improves the power response speed by 30% and the frequency response speed by 47%. The controller has stronger robustness and adaptability for wind-hydro hybrid power system.

Key words

reinforcement learning / deep neural networks / power quality / wind-hydro hybrid power system / intelligent control

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Chen Diyi, Dong Wenhui, Yuan Yichen, Xu Beibei. ADJUSTMENT PERFORMANCE ANALYSIS OF WIND-HYDRO HYBRID POWER SYSTEM UNDER MINI NEURAL NETWORK REINFORCEMENT LEARNING[J]. Acta Energiae Solaris Sinica. 2023, 44(10): 329-338 https://doi.org/10.19912/j.0254-0096.tynxb.2022-0869

References

[1] 卫志农, 裴蕾, 陈胜, 等. 高比例新能源交直流混合配电网优化运行与安全分析研究综述[J]. 电力自动化设备, 2021, 41(9): 85-94.
WEI Z N, PEI L, CHEN S, et al.Review on optimal operation and safety analysis of AC/DC hybrid distribution network with high proportion of renewable energy[J]. Electric power automation equipment, 2021, 41(9): 85-94.
[2] 叶林, 屈晓旭, 么艳香, 等. 风光水多能互补发电系统日内时间尺度运行特性分析[J]. 电力系统自动化, 2018, 42(4): 158-164.
YE L, QU X X, YAO Y X, et al.Analysis on intraday operation characteristics of hybrid wind-solar-hydro power generation system[J]. Automation of electric power systems, 2018, 42(4): 158-164.
[3] PIRAISOODI T, MARIA SILUVAIRAJ W I, KAPPUVA M A K. Multi-objective robust fuzzy fractional order proportional-integral-derivative controller design for nonlinear hydraulic turbine governing system using evolutionary computation techniques[J]. Expert systems, 2019, 36(2): 1-15.
[4] GIL-GONZÁLEZ W J, GARCES A, FOSSO O B, et al. Passivity-based control of power systems considering hydro-turbine with surge tank[J]. IEEE transactions on power systems, 2020, 35(3): 2002-2011.
[5] ZHOU J Z, ZHAO Z G, ZHANG C, et al.A real-time accurate model and its predictive fuzzy PID controller for pumped storage unit via error compensation[J]. Energies, 2017, 11(1): 35-46.
[6] 胡林, 申建建, 曾云, 等. 水电机组PID功率调节对异步送端电网的频率稳定性影响[J]. 电力自动化设备, 2019, 39(11): 217-223.
HU L, SHEN J J, ZENG Y, et al.Impact of PID power regulation of hydroelectric unit on frequency stability for asynchronous sending-end power grid[J]. Electric power automation equipment, 2019, 39(11): 217-223.
[7] 倪爽, 崔承刚, 杨宁, 等. 基于深度强化学习的配电网多时间尺度在线无功优化[J]. 电力系统自动化, 2021, 45(10): 77-85.
NI S, CUI C G, YANG N, et al.Multi-time-scale online optimization for reactive power of distribution network based on deep reinforcement learning[J]. Automation of electric power systems, 2021, 45(10): 77-85.
[8] GUPTA P, PAL A, VITTAL V.Coordinated wide-area damping control using deep neural networks and reinforcement learning[J]. IEEE transactions on power systems, 2022, 37(1): 365-376.
[9] LEI L, TAN Y, DAHLENBURG G, et al.Dynamic energy dispatch based on deep reinforcement learning in IoT-driven smart isolated microgrids[J]. IEEE internet of things journal, 2021, 8(10): 7938-7953.
[10] AL-SAFFAR M, MUSILEK P.Distributed optimization for distribution grids with stochastic DER using multi-agent deep reinforcement learning[J]. IEEE access, 2021, 9: 63059-63072.
[11] ZHANG Z D, ZHANG D X, QIU R C.Deep reinforcement learning for power system applications: an overview[J]. CSEE journal of power and energy systems, 2019, 6(1): 213-225.
[12] 席磊, 余璐, 付一木, 等. 基于探索感知思维深度强化学习的自动发电控制[J]. 中国电机工程学报, 2019, 39(14): 4150-4162.
XI L, YU L, FU Y M, et al.Automatic generation control based on deep reinforcement learning with exploration awareness[J]. Proceedings of the CSEE, 2019, 39(14): 4150-4162.
[13] 赵源筱. 含高比例可再生能源电力系统自动发电控制策略研究[D]. 杭州: 浙江大学, 2021.
ZHAO Y X.Research on automatic generation control strategies for large scale renewable energy integrated power systems[D]. Hangzhou: Zhejiang University, 2021.
[14] 孙文涛, 刘涤尘, 赵洁, 等. 适用于电力系统稳定分析的冲击式水轮机调节系统动态模型[J]. 电网技术, 2014, 38(8): 2071-2075.
SUN W T, LIU D C, ZHAO J, et al.Dynamic models of pelton turbine and speed governing system suitable for power system stability analysis[J]. Power system technology, 2014, 38(8): 2071-2075.
[15] 戴理韬, 高剑, 黄守道, 等. 变速恒频水力发电技术及其发展[J]. 电力系统自动化, 2020, 44(24):169-177.
DAI L T, GAO J, HUANG S D, et al.Variable-speed constant-frequency hydropower generation technology and its development[J]. Automation of electric power systems, 2020, 44(24): 169-177.
[16] 吴嵌嵌, 张雷克, 马震岳. 水电站水机电-结构系统动力耦联模型研究及数值模拟[J]. 振动与冲击, 2017, 36(16): 1-10.
WU Q Q, ZHANG L K, MA Z Y.Model analysis and numerical simulation of a dynamic coupled hydraulic-mechanical-electric-structural system for a hydropower station[J]. Journal of vibration and shock, 2017, 36(16): 1-10.
[17] 薛宇, 任永峰, 胡志帅, 等. 基于九开关变换器的直驱-双馈分散式风电系统控制策略[J]. 电力系统自动化, 2022, 46(7): 152-159.
XUE Y, REN Y F, HU Z S, et al.Control strategy of direct-drive-doubly-fed distributed wind power system based on nine-switch converter[J]. Automation of electric power systems, 2022, 46(7): 152-159.
[18] YU L, SUN Y, XU Z B, et al.Multi-agent deep reinforcement learning for HVAC control in commercial buildings[J]. IEEE transactions on smart grid, 2021, 12(1): 407-419.
[19] SILVER D, LEVER G, HEESS N, et al.Deterministic policy gradient algorithms[C]//Proceedings of the 31st International Conference on International Conference on Machine Learning Volume 32, Beijing, China, 2014: 387-395.
[20] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB/OL].2015: arXiv: 1509.02971. https://arxiv.org/abs/1509.02971.PDF.
[21] IANDOLA F N, HAN S, MOSKEWICZ M W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size[EB/OL].2016: arXiv: 1602.07360. https://arxiv.org/abs/1602.07360.
[22] 李嘉文, 余涛, 张孝顺, 等. 基于改进深度确定性梯度算法的AGC发电功率指令分配方法[J]. 中国电机工程学报, 2021, 41(21): 7198-7212.
LI J W, YU T, ZHANG X S, et al.AGC power generation command allocation method based on improved deep deterministic policy gradient algorithm[J]. Proceedings of the CSEE, 2021, 41(21): 7198-7212.