IDEAS home Printed from https://ideas.repec.org/a/sae/risrel/v234y2020i1p52-62.html
   My bibliography  Save this article

A reinforcement learning approach to optimal part flow management for gas turbine maintenance

Author

Listed:
  • Michele Compare
  • Luca Bellani
  • Enrico Cobelli
  • Enrico Zio
  • Francesco Annunziata
  • Fausto Carlevaro
  • Marzia Sepe

Abstract

We consider the maintenance process of gas turbines used in the Oil and Gas industry: the capital parts are first removed from the gas turbines and replaced by parts of the same type taken from the warehouse; then, they are repaired at the workshop and returned to the warehouse for use in future maintenance events. Experience-based rules are used to manage the flow of the parts for a profitable gas turbine operation. In this article, we formalize the part flow management as a sequential decision problem and propose reinforcement learning for its solution. An application to a scaled-down case study derived from real industrial practice shows that reinforcement learning can find policies outperforming those based on experience-based rules.

Suggested Citation

  • Michele Compare & Luca Bellani & Enrico Cobelli & Enrico Zio & Francesco Annunziata & Fausto Carlevaro & Marzia Sepe, 2020. "A reinforcement learning approach to optimal part flow management for gas turbine maintenance," Journal of Risk and Reliability, , vol. 234(1), pages 52-62, February.
    • Handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62
    DOI: 10.1177/1748006X19869750
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/1748006X19869750
    Download Restriction: no
    File URL: https://libkey.io/10.1177/1748006X19869750?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jin, Tongdan & Tian, Zhigang & Xie, Min, 2015. "A game-theoretical approach for optimizing maintenance, spares and service capacity in performance contracting," International Journal of Production Economics, Elsevier, vol. 161(C), pages 31-43.
    2. Wang, Wenbin, 2010. "A model for maintenance service contract design, negotiation and optimization," European Journal of Operational Research, Elsevier, vol. 201(1), pages 239-246, February.
    3. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    4. Qinming Liu & Ming Dong & Wenyuan Lv & Chunming Ye, 2019. "Manufacturing system maintenance based on dynamic programming model with prognostics information," Journal of Intelligent Manufacturing, Springer, vol. 30(3), pages 1155-1173, March.
    5. Olde Keizer, Minou C.A. & Teunter, Ruud H. & Veldman, Jasper, 2017. "Joint condition-based maintenance and inventory optimization for systems with multiple components," European Journal of Operational Research, Elsevier, vol. 257(1), pages 209-222.
    6. Murthy, D. N. P. & Asgharizadeh, E., 1999. "Optimal decision making in a maintenance service operation," European Journal of Operational Research, Elsevier, vol. 116(2), pages 259-273, July.
    7. Giannoccaro, Ilaria & Pontrandolfo, Pierpaolo, 2002. "Inventory management in supply chains: a reinforcement learning approach," International Journal of Production Economics, Elsevier, vol. 78(2), pages 153-161, July.
    8. Jaturonnatee, J. & Murthy, D.N.P. & Boondiskulchok, R., 2006. "Optimal preventive maintenance of leased equipment with corrective minimal repairs," European Journal of Operational Research, Elsevier, vol. 174(1), pages 201-215, October.
    9. Kurz, Julian, 2016. "Capacity planning for a maintenance service provider with advanced information," European Journal of Operational Research, Elsevier, vol. 251(2), pages 466-477.
    10. Van Horenbeek, Adriaan & Buré, Jasmine & Cattrysse, Dirk & Pintelon, Liliane & Vansteenwegen, Pieter, 2013. "Joint maintenance and inventory optimization systems: A review," International Journal of Production Economics, Elsevier, vol. 143(2), pages 499-508.
    11. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    12. Stephane R. A. Barde & Soumaya Yacout & Hayong Shin, 2019. "Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks," Journal of Intelligent Manufacturing, Springer, vol. 30(1), pages 147-161, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barlow, E. & Bedford, T. & Revie, M. & Tan, J. & Walls, L., 2021. "A performance-centred approach to optimising maintenance of complex systems," European Journal of Operational Research, Elsevier, vol. 292(2), pages 579-595.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jackson, Canek & Pascual, Rodrigo & Mac Cawley, Alejandro & Godoy, Sergio, 2023. "Product–service system negotiation in aircraft lease contracts with option of disagreement," Journal of Air Transport Management, Elsevier, vol. 107(C).
    2. Huber, Sebastian & Spinler, Stefan, 2012. "Pricing of full-service repair contracts," European Journal of Operational Research, Elsevier, vol. 222(1), pages 113-121.
    3. Wang, Yukun & Gao, Weizheng & Li, Xiaopeng & Liu, Yiliu, 2024. "Joint optimization of performance-based contracting, condition-based maintenance and spare parts inventory for degrading production systems," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    4. Zhang, Xiaohong & Liao, Haitao & Zeng, Jianchao & Shi, Guannan & Zhao, Bing, 2021. "Optimal Condition-based Opportunistic Maintenance and Spare Parts Provisioning for a Two-unit System using a State Space Partitioning Approach," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    5. Rahimi-Ghahroodi, S. & Al Hanbali, A. & Zijm, W.H.M. & Timmer, J.B., 2019. "Emergency supply contracts for a service provider with limited local resources," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 445-460.
    6. Michele Compare & Francesco Di Maio & Enrico Zio & Fausto Carlevaro & Sara Mattafirri, 2016. "Improving scheduled maintenance by missing data reconstruction: A double-loop Monte Carlo approach," Journal of Risk and Reliability, , vol. 230(5), pages 502-511, October.
    7. Darghouth, M.N. & Ait-kadi, D. & Chelbi, A., 2017. "Joint optimization of design, warranty and price for products sold with maintenance service contracts," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 197-208.
    8. Liu, Xinbao & Yang, Tianji & Pei, Jun & Liao, Haitao & Pohl, Edward A., 2019. "Replacement and inventory control for a multi-customer product service system with decreasing replacement costs," European Journal of Operational Research, Elsevier, vol. 273(2), pages 561-574.
    9. Nguyen, Kim-Anh & Do, Phuc & Grall, Antoine, 2017. "Joint predictive maintenance and inventory strategy for multi-component systems using Birnbaum’s structural importance," Reliability Engineering and System Safety, Elsevier, vol. 168(C), pages 249-261.
    10. Yan, Tao & Lei, Yaguo & Wang, Biao & Han, Tianyu & Si, Xiaosheng & Li, Naipeng, 2020. "Joint maintenance and spare parts inventory optimization for multi-unit systems considering imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    11. Zhang, Xiaohong & Zeng, Jianchao, 2017. "Joint optimization of condition-based opportunistic maintenance and spare parts provisioning policy in multiunit systems," European Journal of Operational Research, Elsevier, vol. 262(2), pages 479-498.
    12. Zhu, Mixin & Zhou, Xiaojun, 2022. "Hypergraph-based joint optimization of spare part provision and maintenance scheduling for serial-parallel multi-station manufacturing systems," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    13. Barlow, E. & Bedford, T. & Revie, M. & Tan, J. & Walls, L., 2021. "A performance-centred approach to optimising maintenance of complex systems," European Journal of Operational Research, Elsevier, vol. 292(2), pages 579-595.
    14. Shi, Zhenyang & Liu, Shaoxuan, 2020. "Optimal inventory control and design refresh selection in managing part obsolescence," European Journal of Operational Research, Elsevier, vol. 287(1), pages 133-144.
    15. Esmaeili, M. & Shamsi Gamchi, N. & Asgharizadeh, E., 2014. "Three-level warranty service contract among manufacturer, agent and customer: A game-theoretical approach," European Journal of Operational Research, Elsevier, vol. 239(1), pages 177-186.
    16. Feng Tian & Peng Sun & Izak Duenyas, 2021. "Optimal Contract for Machine Repair and Maintenance," Operations Research, INFORMS, vol. 69(3), pages 916-949, May.
    17. de Jonge, Bram & Scarf, Philip A., 2020. "A review on maintenance optimization," European Journal of Operational Research, Elsevier, vol. 285(3), pages 805-824.
    18. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    19. Sgarbossa, Fabio & Peron, Mirco & Lolli, Francesco & Balugani, Elia, 2021. "Conventional or additive manufacturing for spare parts management: An extensive comparison for Poisson demand," International Journal of Production Economics, Elsevier, vol. 233(C).
    20. Zheng, Meimei & Ye, Hongqing & Wang, Dong & Pan, Ershun, 2021. "Joint Optimization of Condition-Based Maintenance and Spare Parts Orders for Multi-Unit Systems with Dual Sourcing," Reliability Engineering and System Safety, Elsevier, vol. 210(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.