针对可再生能源发电过程中间歇性、波动性出力带来的系统调度优化问题,提出一种考虑安稳风险成本的可再生能源发电优化调度算法。首先,以可再生能源发电系统的发电成本、安稳成本以及可再生能源消纳比例为优化目标,建立可再生能源发电多目标优化调度模型。其次,基于深度强化学习的原理,提出可再生能源发电系统调度模型优化求解框架,并结合马尔可夫决策过程,对可再生能源发电多目标优化调度问题进行转换描述,建立基于深度强化学习的模型求解流程。然后,结合PandaPower、OpenAI Gym和Stable-Baselines等开源软件包,搭建基于深度强化学习的可再生能源发电调度开源平台（SEDRL）,依据实际电网参数构建系统优化调度算法程序,并利用不同深度强化学习算法进行测试。最后,基于IEEE 118节点系统搭建测试算例进行验证,结果表明,开源平台具有良好的扩展性,且适用于不同场景下可再生能源发电调度策略的生成。
Abstract
In response to the system scheduling optimization problem caused by the intermittent and fluctuating output of renewable energy generation, an optimal scheduling algorithm of renewable energy power generation considering security and stability risk cost is proposed. Firstly, a multi-objective optimal scheduling model of renewable energy generation is established with the power generation cost, the stability cost, and the proportion of renewable energy consumption as optimization objectives. Secondly, based on the principles of deep reinforcement learning, an optimization solution framework for the scheduling model of renewable energy power generation system is proposed. Combined with the Markov Decision Process （MDP）, the multi-objective optimization scheduling problem of renewable energy generation is transformed and described to establish a solution process for the model based on deep reinforcement learning. Then, an open-source platform for scheduling of renewable energy with deep reinforcement learning （SEDRL） is established using open-source software packages such as PandaPower, OpenAI Gym, and Stable-Baselines. According to the actual power grid parameters, the system optimization scheduling algorithm program is established, and different deep reinforcement learning algorithms are used for verification. Finally, a test example is established based on the IEEE 118-bus system for verification. The results show that the open-source platform has good scalability and is suitable for the generation of renewable energy generation scheduling strategies in different scenarios.
关键词
可再生能源 /
优化调度 /
多目标优化 /
深度强化学习 /
安稳风险
Key words
renewable energy /
optimal scheduling /
multi-objective optimization /
deep reinforcement learning /
security risk
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 罗政杰, 任惠, 辛国雨, 等. 基于模型预测控制的高比例可再生能源电力系统多时间尺度动态可靠优化调度[J]. 太阳能学报, 2024, 45(6): 150-160.
Luo Z J, Ren H, Xin G Y, et al.Multi-time scale dynamic reliable optimal scheduling of power system with high propottion renewable energy based on model predictive control[J]. Acta Energiae Solaris Sinica, 2024, 45(6): 150-160.
[2] Tang Y, Huang Y H, Wang H Z, et al.Framework for artificial intelligence analysis in large-scale power grids based on digital simulation[J]. CSEE Journal of Power and Energy Systems, 2018, 4(4): 459-468.
[3] 罗远翔, 王宇航, 刘铖, 等. 风-光-火-蓄联合系统两阶段优化调度[J]. 太阳能学报, 2023, 44(1): 500-508.
Luo Y X, Wang Y H, Liu C, et al.Two-stage optimal dispatching of wind power-photovoltaic-thermal power-pumped storage combined system[J]. Acta Energiae Solaris Sinica, 2023, 44(1): 500-508.
[4] 付一木, 刘明波. 求解多目标随机动态经济调度问题的场景解耦方法[J]. 电力系统自动化, 2014, 38(9): 34-40.
Fu Y M, Liu M B.Scenario decomposition method for multi-objective stochastic dynamic economical dispatch problem[J]. Automation of Electric Power Systems, 2014, 38(9): 34-40.
[5] 李咸善, 胡家旗, 张远航, 等. 风光水储联合体多时间尺度市场化运营调度策略[J]. 中国电机工程学报, 2025, 45(22): 8879-8892.
Li X S, Hu J Q, Zhang Y H, et al. Optimal scheduling strategy for joint operation of wind-solar-water-storage consortium participating in power market under multiple time scales[J]. Proceedings of the CSEE, 2025, 45(22): 8879--8892.
[6] Silver D, Schrittwieser J, Simonyan K, et al.Mastering the game of Go without human knowledge[J]. Nature, 2017, 550(7676): 354-359.
[7] Silver D, Huang A, Maddison C J, et al.Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.
[8] Amarjyoti S. Deep reinforcement learning for robotic manipulation-the state of the art[J]. arXiv preprint arXiv:1701.08878, 2017.
[9] Khooban M H, Gheisarnejad M.A novel deep reinforcement learning controller based type-Ⅱ fuzzy system: frequency regulation in microgrids[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2020, 5(4): 689-699.
[10] 刘晓明, 刘俊, 姚宏伟, 等. 基于VSG的风光水火储系统频率调节深度强化学习方法[J]. 电力系统自动化, 2025, 49(9): 114-124.
Liu X M, Liu J, Yao H W, et al.Reinforcement learning method for frequency adjustment depth of wind, light, water and fire storage system based on VSG[J]. Automation of Electric Power Systems, 2025, 49(9): 114-124.
[11] 周良才, 周毅, 沈维健, 等. 基于深度强化学习的新型电力系统无功电压优化控制[J]. 电测与仪表, 2024, 61(9): 182-189.
Zhou L C, Zhou Y, Shen W J, et al.Reactive voltage optimization control of novel power system based on deep reinforcement learning[J]. Electrical Measurement & Instrumentation, 2024, 61(9): 182-189.
[12] 王珂, 姚建国, 余佩遥, 等. 基于深度强化学习的电网前瞻调度智能决策架构及关键技术初探[J]. 中国电机工程学报, 2022, 42(15): 5430-5439.
Wang K, Yao J G, Yu P Y, et al.Architecture and key technologies of intelligent decision-making of power grid look-ahead dispatch based on deep reinforcement learning[J]. Proceedings of the CSEE, 2022, 42(15): 5430-5439.
[13] Yan Z M, XU Y.Real-time optimal power flow: a Lagrangian based deep reinforcement learning approach[J]. IEEE Transactions on Power Systems, 2020, 35(4): 3270-3273.
[14] Woo J H, Wu L, Park J B, et al.Real-time optimal power flow using twin delayed deep deterministic policy gradient algorithm[J]. IEEE Access, 2020, 8: 213611-213618.
[15] 董文康, 吴雨芯, 姚琦, 等. 基于深度强化学习的海上风电机组状态维护与备件库存联合优化[J]. 太阳能学报, 2023, 44(12): 190-199.
Dong W K, Wu Y X, Yao Q, et al.Joint optimization of state maintenance and spare parts inventory of offshore wind turbines based on deep reinforcement learning[J]. Acta Energiae Solaris Sinica, 2023, 44(12): 190-199.
[16] 张郁, 苑波, 黄石成, 等. 考虑高比例可再生能源接入的有源配电网经济调度策略研究[J]. 电测与仪表, 2025, 62(1): 158-166.
Zhang Y, Yuan B, Huang S C, et al.Research on economic dispatching strategy for active distribution network considering high penetration of renewable energy source[J]. Electrical Measurement & Instrumentation, 2025, 62(1): 158-166.
[17] 张波, 高远, 李铁成, 等. 考虑光伏电源可靠性的新能源配电网数据驱动无功电压优化控制[J]. 中国电机工程学报, 2024, 44(15): 5934-5947.
Zhang B, Gao Y, Li T C, et al.Data-driven voltage/var optimization control of active distribution network considering the reliability of photovoltaic power supply[J]. Proceedings of the CSEE, 2024, 44(15): 5934-5947.
基金
国家自然科学基金青年科学基金（52307095）
PDF(1095 KB)
