IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v219y2022ics0951832021006773.html
   My bibliography  Save this article

Maintenance policy analysis of the regenerative air heater system using factored POMDPs

Author

Listed:
  • Kıvanç, Ä°pek
  • Özgür-Ãœnlüakın, Demet
  • Bilgiç, Taner

Abstract

Maintenance optimization of multi-component systems is a difficult problem. Partially Observable Markov Decision Processes (POMDPs) are powerful tools for such problems under uncertainty in stochastic environments. In this study, the main POMDP solution approaches and solvers are surveyed. Then, based on experimental models with different complexities in the size of the system space, selected POMDP solvers using different representation patterns for modeling and different procedures for updating the value function while solving are compared. Furthermore, to show that factored representations are advantageous in modeling and solving the maintenance problem of multi-component systems where there exist also stochastic dependencies among the components, the maintenance problem of the one-line regenerative air heater system available in thermal power plants is modeled and solved with factored POMDPs. In-depth sensitivity analyses are performed on the obtained policy. The results show that factored POMDPs enable compact modeling, efficient policy generation and practical policy analysis for the tackled problem. Furthermore, they also motivate the use of factored POMDPs in the generation and analysis of maintenance policies for similar multi-component systems.

Suggested Citation

  • Kıvanç, Ä°pek & Özgür-Ãœnlüakın, Demet & Bilgiç, Taner, 2022. "Maintenance policy analysis of the regenerative air heater system using factored POMDPs," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    • Handle: RePEc:eee:reensy:v:219:y:2022:i:c:s0951832021006773
    DOI: 10.1016/j.ress.2021.108195
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832021006773
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2021.108195?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Özgür-Ünlüakın, Demet & Türkali, Busenur & Karacaörenli, Ayşe & Çağlar Aksezer, S., 2019. "A DBN based reactive maintenance model for a complex system in thermal power plants," Reliability Engineering and System Safety, Elsevier, vol. 190(C), pages 1-1.
    2. Samer Madanat & Moshe Ben-Akiva, 1994. "Optimal Inspection and Repair Policies for Infrastructure Facilities," Transportation Science, INFORMS, vol. 28(1), pages 55-62, February.
    3. Özgür-Ünlüakın, Demet & Bilgiç, Taner, 2017. "Performance analysis of an aggregation and disaggregation solution procedure to obtain a maintenance plan for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 652-662.
    4. van Staden, Heletjé E. & Boute, Robert N., 2021. "The effect of multi-sensor data on condition-based maintenance policies," European Journal of Operational Research, Elsevier, vol. 290(2), pages 585-600.
    5. Papakonstantinou, K.G. & Shinozuka, M., 2014. "Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory," Reliability Engineering and System Safety, Elsevier, vol. 130(C), pages 202-213.
    6. Nguyen, Khanh T. P. & Do, Phuc & Huynh, Khac Tuan & Bérenguer, Christophe & Grall, Antoine, 2019. "Joint optimization of monitoring quality and replacement decisions in condition-based maintenance," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 177-195.
    7. George E. Monahan, 1982. "State of the Art---A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms," Management Science, INFORMS, vol. 28(1), pages 1-16, January.
    8. KarabaÄŸ, Oktay & Eruguz, Ayse Sena & Basten, Rob, 2020. "Integrated optimization of maintenance interventions and spare part selection for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    9. Edward J. Sondik, 1978. "The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs," Operations Research, INFORMS, vol. 26(2), pages 282-304, April.
    10. Papakonstantinou, K.G. & Shinozuka, M., 2014. "Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation," Reliability Engineering and System Safety, Elsevier, vol. 130(C), pages 214-224.
    11. Andriotis, C.P. & Papakonstantinou, K.G., 2021. "Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhao, Yunfei & Smidts, Carol, 2022. "Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states," Reliability Engineering and System Safety, Elsevier, vol. 224(C).
    2. Wang, Jiaxi, 2024. "Maintenance scheduling at high-speed train depots: An optimization approach," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    3. Xu, Jianyu & Liu, Bin & Zhao, Xiujie & Wang, Xiao-Lin, 2024. "Online reinforcement learning for condition-based group maintenance using factored Markov decision processes," European Journal of Operational Research, Elsevier, vol. 315(1), pages 176-190.
    4. Arcieri, Giacomo & Hoelzl, Cyprien & Schwery, Oliver & Straub, Daniel & Papakonstantinou, Konstantinos G. & Chatzi, Eleni, 2023. "Bridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systems," Reliability Engineering and System Safety, Elsevier, vol. 239(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. KarabaÄŸ, Oktay & Eruguz, Ayse Sena & Basten, Rob, 2020. "Integrated optimization of maintenance interventions and spare part selection for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    2. Özgür-Ünlüakın, Demet & Bilgiç, Taner, 2017. "Performance analysis of an aggregation and disaggregation solution procedure to obtain a maintenance plan for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 652-662.
    3. Seites-Rundlett, William & Bashar, Mohammad Z. & Torres-Machi, Cristina & Corotis, Ross B., 2022. "Combined evidence model to enhance pavement condition prediction from highly uncertain sensor data," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    4. Tseremoglou, Iordanis & Santos, Bruno F., 2024. "Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    5. Deep, Akash & Zhou, Shiyu & Veeramani, Dharmaraj & Chen, Yong, 2023. "Partially observable Markov decision process-based optimal maintenance planning with time-dependent observations," European Journal of Operational Research, Elsevier, vol. 311(2), pages 533-544.
    6. Morato, P.G. & Andriotis, C.P. & Papakonstantinou, K.G. & Rigo, P., 2023. "Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    7. Papakonstantinou, K.G. & Shinozuka, M., 2014. "Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory," Reliability Engineering and System Safety, Elsevier, vol. 130(C), pages 202-213.
    8. Joaquim AP Braga & António R Andrade, 2019. "Optimizing maintenance decisions in railway wheelsets: A Markov decision process approach," Journal of Risk and Reliability, , vol. 233(2), pages 285-300, April.
    9. Memarzadeh, Milad & Pozzi, Matteo & Kolter, J. Zico, 2016. "Hierarchical modeling of systems with similar components: A framework for adaptive monitoring and control," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 159-169.
    10. Özgür-Ünlüakın, Demet & Türkali, Busenur, 2021. "Evaluation of proactive maintenance policies on a stochastically dependent hidden multi-component system using DBNs," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    11. Kim, Seokgoo & Choi, Joo-Ho & Kim, Nam Ho, 2022. "Inspection schedule for prognostics with uncertainty management," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    12. Yanling Chang & Alan Erera & Chelsea White, 2015. "Value of information for a leader–follower partially observed Markov game," Annals of Operations Research, Springer, vol. 235(1), pages 129-153, December.
    13. Lv, Y. & Yan, X.D. & Sun, W. & Gao, Z.Y., 2015. "A risk-based method for planning of bus–subway corridor evacuation under hybrid uncertainties," Reliability Engineering and System Safety, Elsevier, vol. 139(C), pages 188-199.
    14. Hao Zhang, 2010. "Partially Observable Markov Decision Processes: A Geometric Technique and Analysis," Operations Research, INFORMS, vol. 58(1), pages 214-228, February.
    15. Saghafian, Soroush, 2018. "Ambiguous partially observable Markov decision processes: Structural results and applications," Journal of Economic Theory, Elsevier, vol. 178(C), pages 1-35.
    16. Nguyen, Khanh T.P. & Medjaher, Kamal, 2019. "A new dynamic predictive maintenance framework using deep learning for failure prognostics," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 251-262.
    17. Memarzadeh, Milad & Pozzi, Matteo, 2016. "Value of information in sequential decision making: Component inspection, permanent monitoring and system-level scheduling," Reliability Engineering and System Safety, Elsevier, vol. 154(C), pages 137-151.
    18. Ferreira Neto, Waldomiro Alves & Virgínio Cavalcante, Cristiano Alexandre & Do, Phuc, 2024. "Deep reinforcement learning for maintenance optimization of a scrap-based steel production line," Reliability Engineering and System Safety, Elsevier, vol. 249(C).
    19. Ricardo Montoya & Oded Netzer & Kamel Jedidi, 2010. "Dynamic Allocation of Pharmaceutical Detailing and Sampling for Long-Term Profitability," Marketing Science, INFORMS, vol. 29(5), pages 909-924, 09-10.
    20. Nozhati, Saeed & Sarkale, Yugandhar & Chong, Edwin K.P. & Ellingwood, Bruce R., 2020. "Optimal stochastic dynamic scheduling for managing community recovery from natural hazards," Reliability Engineering and System Safety, Elsevier, vol. 193(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:219:y:2022:i:c:s0951832021006773. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.