IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v377y2025ipas0306261924018142.html
   My bibliography  Save this article

A deep reinforcement learning ensemble for maintenance scheduling in offshore wind farms

Author

Listed:
  • Lee, Namkyoung
  • Woo, Joohyun
  • Kim, Sungryul

Abstract

Offshore wind energy, a cornerstone of sustainable power generation, faces escalating operational challenges as farms expand to harness cost efficiencies, including the imperative to counteract power fluctuations caused by wake effects and weather volatility. This study introduces a domain-informed Deep Q-Network (DQN) framework, engineered to optimize the allocation of maintenance resources and the strategic selection of maintenance tasks, resulting in an 11.1% increase in power generation compared to default wind conditions. By incorporating multiple wake model for enhanced decision-making accuracy, the scheduling dilemma is formulated as Markov Decision Processes (MDPs) to navigate the complexities of maintenance scheduling. A notable innovation is the integration of convolutional layers, which expedite algorithmic convergence. These results underscore the significant potential of our model to improve operational productivity in large-scale offshore wind farms.

Suggested Citation

  • Lee, Namkyoung & Woo, Joohyun & Kim, Sungryul, 2025. "A deep reinforcement learning ensemble for maintenance scheduling in offshore wind farms," Applied Energy, Elsevier, vol. 377(PA).
    • Handle: RePEc:eee:appene:v:377:y:2025:i:pa:s0306261924018142
    DOI: 10.1016/j.apenergy.2024.124431
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924018142
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.124431?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Shakoor, Rabia & Hassan, Mohammad Yusri & Raheem, Abdur & Wu, Yuan-Kang, 2016. "Wake effect modeling: A review of wind farm layout optimization using Jensen׳s model," Renewable and Sustainable Energy Reviews, Elsevier, vol. 58(C), pages 1048-1059.
    2. John Mathews & Elizabeth Thurbon & Sung-Young Kim & Hao Tan, 2023. "Gone with the wind: how state power and industrial policy in the offshore wind power sector are blowing away the obstacles to East Asia’s green energy transition," Review of Evolutionary Political Economy, Springer, vol. 4(1), pages 27-48, April.
    3. Zhang, Chen & Yang, Tao, 2021. "Optimal maintenance planning and resource allocation for wind farms based on non-dominated sorting genetic algorithm-ΙΙ," Renewable Energy, Elsevier, vol. 164(C), pages 1540-1549.
    4. Zhou, Yifan & Miao, Jindan & Yan, Bin & Zhang, Zhisheng, 2020. "Bio-objective long-term maintenance scheduling for wind turbines in multiple wind farms," Renewable Energy, Elsevier, vol. 160(C), pages 1136-1147.
    5. Luca Pinciroli & Piero Baraldi & Guido Ballabio & Michele Compare & Enrico Zio, 2021. "Deep Reinforcement Learning Based on Proximal Policy Optimization for the Maintenance of a Wind Farm with Multiple Crews," Energies, MDPI, vol. 14(20), pages 1-17, October.
    6. Bengio, Yoshua & Lodi, Andrea & Prouvost, Antoine, 2021. "Machine learning for combinatorial optimization: A methodological tour d’horizon," European Journal of Operational Research, Elsevier, vol. 290(2), pages 405-421.
    7. Irawan, Chandra Ade & Ouelhadj, Djamila & Jones, Dylan & Stålhane, Magnus & Sperstad, Iver Bakken, 2017. "Optimisation of maintenance routing and scheduling for offshore wind farms," European Journal of Operational Research, Elsevier, vol. 256(1), pages 76-89.
    8. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    9. Li, Xiaodong & Ouelhadj, Djamila & Song, Xiang & Jones, Dylan & Wall, Graham & Howell, Kerry E. & Igwe, Paul & Martin, Simon & Song, Dongping & Pertin, Emmanuel, 2016. "A decision support system for strategic maintenance planning in offshore wind farms," Renewable Energy, Elsevier, vol. 99(C), pages 784-799.
    10. Gutierrez-Alcoba, A. & Hendrix, E.M.T. & Ortega, G. & Halvorsen-Weare, E.E. & Haugland, D., 2019. "On offshore wind farm maintenance scheduling for decision support on vessel fleet composition," European Journal of Operational Research, Elsevier, vol. 279(1), pages 124-131.
    11. Christopher Jung & Dirk Schindler, 2022. "Development of onshore wind turbine fleet counteracts climate change-induced reduction in global capacity factor," Nature Energy, Nature, vol. 7(7), pages 608-619, July.
    12. Shields, Matt & Beiter, Philipp & Nunemaker, Jake & Cooperman, Aubryn & Duffy, Patrick, 2021. "Impacts of turbine and plant upsizing on the levelized cost of energy for offshore wind," Applied Energy, Elsevier, vol. 298(C).
    13. Meng, Fanyi & Bai, Yang & Jin, Jingliang, 2021. "An advanced real-time dispatching strategy for a distributed energy system based on the reinforcement learning algorithm," Renewable Energy, Elsevier, vol. 178(C), pages 13-24.
    14. de Jonge, Bram & Scarf, Philip A., 2020. "A review on maintenance optimization," European Journal of Operational Research, Elsevier, vol. 285(3), pages 805-824.
    15. Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    16. Richard Bellman, 1954. "On some applications of the theory of dynamic programming to logistics," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 1(2), pages 141-153, June.
    17. Yang, Kyoungboo & Kwak, Gyeongil & Cho, Kyungho & Huh, Jongchul, 2019. "Wind farm layout optimization for wake effect uniformity," Energy, Elsevier, vol. 183(C), pages 983-995.
    18. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    19. Ryan Wiser & Joseph Rand & Joachim Seel & Philipp Beiter & Erin Baker & Eric Lantz & Patrick Gilman, 2021. "Expert elicitation survey predicts 37% to 49% declines in wind energy costs by 2050," Nature Energy, Nature, vol. 6(5), pages 555-565, May.
    20. Tian, Qun & Huang, Gang & Hu, Kaiming & Niyogi, Dev, 2019. "Observed and global climate model based changes in wind power potential over the Northern Hemisphere during 1979–2016," Energy, Elsevier, vol. 167(C), pages 1224-1235.
    21. Rocchetta, R. & Bellani, L. & Compare, M. & Zio, E. & Patelli, E., 2019. "A reinforcement learning framework for optimal operation and maintenance of power grids," Applied Energy, Elsevier, vol. 241(C), pages 291-301.
    22. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    23. Caralis, George & Diakoulaki, Danae & Yang, Peijin & Gao, Zhiqiu & Zervos, Arthouros & Rados, Kostas, 2014. "Profitability of wind energy investments in China using a Monte Carlo approach for the treatment of uncertainties," Renewable and Sustainable Energy Reviews, Elsevier, vol. 40(C), pages 224-236.
    24. Mittal, Prateek & Kulkarni, Kedar & Mitra, Kishalay, 2016. "A novel hybrid optimization methodology to optimize the total number and placement of wind turbines," Renewable Energy, Elsevier, vol. 86(C), pages 133-147.
    25. John Mathews & Elizabeth Thurbon & Sung-Young Kim & Hao Tan, 2023. "Correction: Gone with the wind: how state power and industrial policy in the offshore wind power sector are blowing away the obstacles to East Asia’s green energy transition," Review of Evolutionary Political Economy, Springer, vol. 4(1), pages 191-192, April.
    26. Malte Jansen & Iain Staffell & Lena Kitzing & Sylvain Quoilin & Edwin Wiggelinkhuizen & Bernard Bulder & Iegor Riepin & Felix Müsgens, 2020. "Offshore wind competitiveness in mature markets without subsidy," Nature Energy, Nature, vol. 5(8), pages 614-622, August.
    27. Stock-Williams, Clym & Swamy, Siddharth Krishna, 2019. "Automated daily maintenance planning for offshore wind farms," Renewable Energy, Elsevier, vol. 133(C), pages 1393-1403.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Qin & Liu, Yu & Xiang, Yisha & Xiahou, Tangfan, 2024. "Reinforcement learning in reliability and maintenance optimization: A tutorial," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    2. Shuo-Yan Chou & Xuan Loc Pham & Thi Anh Tuyet Nguyen & Tiffany Hui-Kuang Yu, 2023. "Optimal maintenance planning with special emphasis on deterioration process and vessel routing for offshore wind systems," Energy & Environment, , vol. 34(4), pages 739-763, June.
    3. Pinciroli, Luca & Baraldi, Piero & Zio, Enrico, 2023. "Maintenance optimization in industry 4.0," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    4. Pryor, Sara C. & Barthelmie, Rebecca J., 2024. "Wind shadows impact planning of large offshore wind farms," Applied Energy, Elsevier, vol. 359(C).
    5. Guan, Xiaoshu & Xiang, Zhengliang & Bao, Yuequan & Li, Hui, 2022. "Structural dominant failure modes searching method based on deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    6. Pedersen, Jaap & Weinand, Jann Michael & Syranidou, Chloi & Rehfeldt, Daniel, 2024. "An efficient solver for large-scale onshore wind farm siting including cable routing," European Journal of Operational Research, Elsevier, vol. 317(2), pages 616-630.
    7. Scarf, P.A. & Cavalcante, C.A.V. & Lopes, R.S., 2019. "Delay-time modelling of a critical system subject to random inspections," European Journal of Operational Research, Elsevier, vol. 278(3), pages 772-782.
    8. Pinciroli, Luca & Baraldi, Piero & Ballabio, Guido & Compare, Michele & Zio, Enrico, 2022. "Optimization of the Operation and Maintenance of renewable energy systems by Deep Reinforcement Learning," Renewable Energy, Elsevier, vol. 183(C), pages 752-763.
    9. Pliego Marugán, Alberto & Pinar-Pérez, Jesús M. & García Márquez, Fausto Pedro, 2024. "A reinforcement learning agent for maintenance of deteriorating systems with increasingly imperfect repairs," Reliability Engineering and System Safety, Elsevier, vol. 252(C).
    10. Lee, Dongkyu & Song, Junho, 2023. "Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    11. Moreno, Sinvaldo Rodrigues & Pierezan, Juliano & Coelho, Leandro dos Santos & Mariani, Viviana Cocco, 2021. "Multi-objective lightning search algorithm applied to wind farm layout optimization," Energy, Elsevier, vol. 216(C).
    12. Esmaeili Aliabadi, Danial & Chan, Katrina, 2022. "The emerging threat of artificial intelligence on competition in liberalized electricity markets: A deep Q-network approach," Applied Energy, Elsevier, vol. 325(C).
    13. Martinez, A. & Iglesias, G., 2024. "Global wind energy resources decline under climate change," Energy, Elsevier, vol. 288(C).
    14. Wu, Yan & Xia, Tianqi & Wang, Yufei & Zhang, Haoran & Feng, Xiao & Song, Xuan & Shibasaki, Ryosuke, 2022. "A synchronization methodology for 3D offshore wind farm layout optimization with multi-type wind turbines and obstacle-avoiding cable network," Renewable Energy, Elsevier, vol. 185(C), pages 302-320.
    15. Du, Yan & Zandi, Helia & Kotevska, Olivera & Kurte, Kuldeep & Munk, Jeffery & Amasyali, Kadir & Mckee, Evan & Li, Fangxing, 2021. "Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 281(C).
    16. Hughes, Llewelyn & Longden, Thomas, 2024. "Offshore wind power in the Asia-Pacific: Expert elicitation on costs and policies," Energy Policy, Elsevier, vol. 184(C).
    17. Najafi, Seyedvahid & Lee, Chi-Guhn, 2023. "A deep reinforcement learning approach for repair-based maintenance of multi-unit systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    18. Mohammadi, Reza & He, Qing, 2022. "A deep reinforcement learning approach for rail renewal and maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    19. Ren, Zhengru & Verma, Amrit Shankar & Li, Ye & Teuwen, Julie J.E. & Jiang, Zhiyu, 2021. "Offshore wind turbine operations and maintenance: A state-of-the-art review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    20. Guan, Xiaoshu & Sun, Huabin & Hou, Rongrong & Xu, Yang & Bao, Yuequan & Li, Hui, 2023. "A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy," Reliability Engineering and System Safety, Elsevier, vol. 233(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:377:y:2025:i:pa:s0306261924018142. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.