JOINT OPTIMIZATION OF STATE MAINTENANCE AND SPARE PARTS INVENTORY OF OFFSHORE WIND TURBINES BASED ON DEEP REINFORCEMENT LEARNING

Dong Wenkang; Wu Yuxin; Yao Qi; Guo Jun; Zhao Tianyang

doi:10.19912/j.0254-0096.tynxb.2022-1219

PDF(1864 KB)

Acta Energiae Solaris Sinica ›› 2023, Vol. 44 ›› Issue (12) : 190-199. DOI: 10.19912/j.0254-0096.tynxb.2022-1219

JOINT OPTIMIZATION OF STATE MAINTENANCE AND SPARE PARTS INVENTORY OF OFFSHORE WIND TURBINES BASED ON DEEP REINFORCEMENT LEARNING

Dong Wenkang¹, Wu Yuxin², Yao Qi¹, Guo Jun³, Zhao Tianyang¹

Author information +

History +

Abstract

Maintenance and spare parts inventory management are two fundamental sequential procedures in offshore wind farm operation and maintenance （OM）. To improve the OM efficiency of offshore wind turbines （OWTs） in the design life cycle, a joint optimization strategy of condition-based maintenance and spare parts inventory control is proposed. Firstly, the OWT is constructed as a series system of subsystems including blades, gearbox, electrical, yaw,wheel hub, braking, transmission chain, generator, the deterioration process of each subsystem is modeled as a multi-state Markov process, and the coupling model of maintenance and spare parts inventory is established. Secondly, the representation and update method of subsystems deterioration, spare parts inventory are formulated as the framework of deep reinforcement learning. Through the iterative training of the deep networks, the optimal maintenance and spare parts ordering decision making are obtained. Finally, taking an OWT of a practical offshore wind farm as an example, the effectiveness of the proposed joint optimization method is verified. The impacts of the exploration rate and the accessibility of OWTs on the OM cost are further discussed.

Key words

offshore wind turbines / state maintenance / inventory control / reinforcement learning / joint optimization

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Dong Wenkang, Wu Yuxin, Yao Qi, Guo Jun, Zhao Tianyang. JOINT OPTIMIZATION OF STATE MAINTENANCE AND SPARE PARTS INVENTORY OF OFFSHORE WIND TURBINES BASED ON DEEP REINFORCEMENT LEARNING[J]. Acta Energiae Solaris Sinica. 2023, 44(12): 190-199 https://doi.org/10.19912/j.0254-0096.tynxb.2022-1219

References

[1] GWEC. Global offshore wind report[EB/OL].https://gwec.net/wp-content/uploads/2021/09/GWEC-offshore-wind-2021-updated-1.pdf.
[2] REN Z R, VERMA A S, LI Y, et al.Offshore wind turbine operations and maintenance: a state-of-the-art review[J]. Renewable and sustainable energy reviews, 2021, 144: 110886.
[3] ALGERN J, PETERS L, MADLENER R. Economic evaluation of maintenance strategies for offshore wind turbines based on condition monitoring systems[EB/OL]. FCN working papers, 2017-7: https://EconPapers.repec.org/RePEc:ris:fcnwpa:2017_008.
[4] 符杨, 许伟欣, 刘璐洁, 等. 考虑天气因素的海上风电机组预防性机会维护策略优化方法[J]. 中国电机工程学报, 2018, 38(20): 5947-5956.
FU Y, XU W X, LIU L J, et al.Optimization of preventive opportunistic maintenance strategy for offshore wind turbine considering weather conditions[J]. Proceedings of the CSEE, 2018, 38(20): 5947-5956.
[5] LI M X, WANG M, KANG J C, et al.An opportunistic maintenance strategy for offshore wind turbine system considering optimal maintenance intervals of subsystems[J]. Ocean engineering, 2020, 216: 108067.
[6] ZHU W J, CASTANIER B, BETTAYEB B.A dynamic programming-based maintenance model of offshore wind turbine considering logistic delay and weather condition[J]. Reliability engineering & system safety, 2019, 190: 106512.
[7] TIAN Z G, JIN T D, WU B R, et al.Condition based maintenance optimization for wind power generation systems under continuous monitoring[J]. Renewable energy, 2011, 36(5): 1502-1509.
[8] BYON E, DING Y.Season-dependent condition-based maintenance for a wind turbine using a partially observed Markov decision process[J]. IEEE transactions on power systems, 2010, 25: 1823-1834.
[9] ZHOU P, YIN P T.An opportunistic condition-based maintenance strategy for offshore wind farm based on predictive analytics[J]. Renewable and sustainable energy reviews, 2019, 109: 1-9.
[10] 黄玲玲, 符杨, 任浩瀚, 等. 基于状态信息的风电机组维护研究综述[J]. 中国电机工程学报, 2020, 40(21): 7065-7078.
HUANG L L, FU Y, REN H H, et al.Review of wind turbine maintenance based on condition monitoring systems[J]. Proceedings of the CSEE, 2020, 40(21): 7065-7078.
[11] ZHU X Y, WANG J, COIT D W.Joint optimization of spare part supply and opportunistic condition-based maintenance for onshore wind farms considering maintenance route[J]. IEEE transactions on engineering management, 2022, 262(2): 479-498.
[12] 胡迪, 高庆水, 张楚, 等. 基于系统动力学的风力机备件需求预测研究[J]. 太阳能学报, 2019, 40(3): 666-672.
HU D, GAO Q S, ZHANG C, et al.Research on demand forecast of spare parts of wind turbine based on system dynamics[J]. Acta energiae solaris sinica, 2019, 40(3): 666-672.
[13] 张再峰, 张琛, 郭盛, 等. 基于周期性预防维修的风电场备件成本分析[J]. 太阳能学报, 2018, 39(5): 1434-1439.
ZHANG Z F, ZHANG C, GUO S, et al.Analysis of spare parts cost for wind farm based on periodic preventive maintenance[J]. Acta energiae solaris sinica, 2018, 39(5): 1434-1439.
[14] 赵洪山, 刘宏杨, 宋鹏, 等. 风电机组大部件的备品备件区域库存优化控制策略[J]. 可再生能源, 2018, 36(3): 422-428.
ZHAO H S, LIU H Y, SONG P, et al.Regional inventory optimization control strategy of spare parts for big parts of wind turbines[J]. Renewable energy resources, 2018, 36(3): 422-428.
[15] NEVES-MOREIRA F, VELDMAN J, TEUNTER R H.Service operation vessels for offshore wind farm maintenance: optimal stock levels[J]. Renewable and sustainable energy reviews, 2021, 146: 111158.
[16] 韩梦莹, 杨建华. 考虑不完美检测的两阶段点检与备件订购策略联合优化[J]. 运筹与管理, 2021, 30(3): 27-34.
HAN M Y, YANG J H.Joint optimization of two-phase inspection and spare parts ordering policies considering imperfect inspection[J]. Operations research and management science, 2021, 30(3): 27-34.
[17] 王孟雅, 陈震, 潘尔顺. 基于马尔科夫决策的冷贮备串联系统状态维修与备件联合优化[J]. 工业工程与管理, 2022, 27(6): 14-23.
WANG M Y, CHEN Z, PAN E S.Joint optimization of condition-based maintenance and spare parts for cold standby series system considering Markov decision[J]. Industrial engineering and management, 2022, 27(6): 14-23.
[18] 杨建华, 韩梦莹. 视情维修条件下k/N(G)系统备件供需联合优化[J]. 系统工程与电子技术, 2019, 41(9): 2148-2156.
YANG J H, HAN M Y.Joint optimization of spare parts supply-demand for k/N(G) system under condition-based maintenance[J]. Systems engineering and electronics, 2019, 41(9): 2148-2156.
[19] ZHENG M M, YE H Q, WANG D, et al.Joint optimization of condition-based maintenance and spare parts orders for multi-unit systems with dual sourcing[J]. Reliability engineering & system safety, 2021, 210: 107512.
[20] ANG J, ZHU X Y.Joint optimization of condition-based maintenance and inventory control for a k-out-of-n:F system of multi-state degrading components[J]. European journal of operational research, 2021, 290(2): 514-529.
[21] KUSHWAHA A, GOPAL M, SINGH B.Q-learning based maximum power extraction for wind energy conversion system with variable wind speed[J]. IEEE transactions on energy conversion, 2020, 35(3): 1160-1170.
[22] LAMBERT N O, SCHINDLER C B, DREW D S, et al.Nonholonomic yaw control of an underactuated flying robot with model-based reinforcement learning[J]. IEEE robotics and automation letters, 2021, 6(2): 455-461.
[23] ZHAO H, ZHAO J H, QIU J, et al.Cooperative wind farm control with deep reinforcement learning and knowledge-assisted learning[J]. IEEE transactions on industrial informatics, 2020, 16(11): 6912-6921.
[24] RAUSAND M, BARROS A, HOYLAND A.System reliability theory: models, statistical methods, and applications[M]. Norway: John Wiley & Sons, 2003.
[25] 白恺, 宋鹏. 风电机组检修决策[M]. 北京: 中国电力出版社, 2021.
BAI K, SONG P.Maintenance decision of wind turbine[M]. Beijing: China Electric Power Press, 2021.
[26] SUTTON R S, BARTO A G.Reinforcement learning: an introduction[M]. 2nd ed. London: MIT Press, 2018.
[27] LE B, ANDREWS J.Modelling wind turbine degradation and maintenance[J]. Wind energy, 2016, 19: 571-591.
[28] CARROLL J, MCDONALD A, MCMILLAN D.Failure rate, repair time and unscheduled O&M cost analysis of offshore wind turbines[J]. Wind energy, 2016, 19(6): 1107-1119.
[29] BVGA. Wind farm costs[EB/OL].https://guidetoanoffsh orewindfarm.com/wind-farm-costs.
[30] SSAI C I, BOSWELL B, DAVIES I J.A Markovian approach for modelling the effects of maintenance on downtime and failure risk of wind turbine components[J]. Renewable energy, 2016, 96(PA): 775-783.