IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v376y2024ipas0306261924015629.html
   My bibliography  Save this article

Optimal operational planning of a bio-fuelled cogeneration plant: Integration of sparse nonlinear dynamics identification and deep reinforcement learning

Author

Listed:
  • Asadzadeh, Seyed Mohammad
  • Andersen, Nils Axel

Abstract

This paper presents a novel data-driven approach for short-term operational planning of a cogeneration plant. The proposed methodology utilizes sparse identification of nonlinear dynamics (SINDy) to extract a dynamic model of heat generation from operational data. This model is then employed to simulate the plant dynamics during the training of a reinforcement learning (RL) agent, enabling online stochastic optimization of the production plan in real-time. The incorporation of SINDy enhances the accuracy of capturing the plant's nonlinear dynamics and significantly improves the computational speed of plant simulations, enabling efficient RL agent training within a reasonable timeframe. The performance of operational planning with the RL agent is compared to that of dynamic programming, a widely used method in the literature. The evaluation metric encompasses energy efficiency, unmet demands, and wasted heat. The comparison investigates the effectiveness of RL and dynamic programming under various scenarios with different qualities of energy demand forecasts. The RL agent exhibits robustness and notably improves the operational planning performance, particularly when faced with uncertain energy demands in the environment. Furthermore, the findings show that the RL agent, trained on a school building data, could successfully perform planning tasks for a hotel building, indicating the transferability of learned planning knowledge across different cogeneration use cases.

Suggested Citation

  • Asadzadeh, Seyed Mohammad & Andersen, Nils Axel, 2024. "Optimal operational planning of a bio-fuelled cogeneration plant: Integration of sparse nonlinear dynamics identification and deep reinforcement learning," Applied Energy, Elsevier, vol. 376(PA).
  • Handle: RePEc:eee:appene:v:376:y:2024:i:pa:s0306261924015629
    DOI: 10.1016/j.apenergy.2024.124179
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924015629
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2024.124179?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dong, Hongyang & Zhang, Jincheng & Zhao, Xiaowei, 2021. "Intelligent wind farm control via deep reinforcement learning and high-fidelity simulations," Applied Energy, Elsevier, vol. 292(C).
    2. Haase, Patrick & Thomas, Bernd, 2021. "Test and optimization of a control algorithm for demand-oriented operation of CHP units using hardware-in-the-loop," Applied Energy, Elsevier, vol. 294(C).
    3. Bosman, M.G.C. & Bakker, V. & Molderink, A. & Hurink, J.L. & Smit, G.J.M., 2012. "Planning the production of a fleet of domestic combined heat and power generators," European Journal of Operational Research, Elsevier, vol. 216(1), pages 140-151.
    4. Oh, Si-Doek & Lee, Ho-Jun & Jung, Jung-Yeul & Kwak, Ho-Young, 2007. "Optimal planning and economic evaluation of cogeneration system," Energy, Elsevier, vol. 32(5), pages 760-771.
    5. Van de Beld, Bert & Holle, Elmar & Florijn, Jan, 2013. "The use of pyrolysis oil and pyrolysis oil derived fuels in diesel engines for CHP applications," Applied Energy, Elsevier, vol. 102(C), pages 190-197.
    6. You, Minglei & Wang, Qian & Sun, Hongjian & Castro, Iván & Jiang, Jing, 2022. "Digital twins based day-ahead integrated energy system scheduling under load and renewable energy uncertainties," Applied Energy, Elsevier, vol. 305(C).
    7. Xu Chen & Shuai Fang & Kangji Li, 2023. "Reinforcement-Learning-Based Multi-Objective Differential Evolution Algorithm for Large-Scale Combined Heat and Power Economic Emission Dispatch," Energies, MDPI, vol. 16(9), pages 1-23, April.
    8. Hua, Weiqi & Stephen, Bruce & Wallom, David C.H., 2023. "Digital twin based reinforcement learning for extracting network structures and load patterns in planning and operation of distribution systems," Applied Energy, Elsevier, vol. 342(C).
    9. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    10. Zhang, Guangming & Zhang, Chao & Wang, Wei & Cao, Huan & Chen, Zhenyu & Niu, Yuguang, 2023. "Offline reinforcement learning control for electricity and heat coordination in a supercritical CHP unit," Energy, Elsevier, vol. 266(C).
    11. Rong, Aiying & Hakonen, Henri & Lahdelma, Risto, 2008. "A variant of the dynamic programming algorithm for unit commitment of combined heat and power systems," European Journal of Operational Research, Elsevier, vol. 190(3), pages 741-755, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hasith Jayasinghe & Kosala Gunawardane & Robert Nicholson, 2025. "Applications of Electrical Load Modelling in Digital Twins of Power Systems," Energies, MDPI, vol. 18(4), pages 1-25, February.
    2. Jochem, Patrick & Schönfelder, Martin & Fichtner, Wolf, 2015. "An efficient two-stage algorithm for decentralized scheduling of micro-CHP units," European Journal of Operational Research, Elsevier, vol. 245(3), pages 862-874.
    3. Lee, Seokhwan & Woo, Sang Hee & Kim, Yongrae & Choi, Young & Kang, Kernyong, 2020. "Combustion and emission characteristics of a diesel-powered generator running with N-butanol/coffee ground pyrolysis oil/diesel blended fuel," Energy, Elsevier, vol. 206(C).
    4. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    5. Zhang, Jincheng & Zhao, Xiaowei, 2021. "Three-dimensional spatiotemporal wind field reconstruction based on physics-informed deep learning," Applied Energy, Elsevier, vol. 300(C).
    6. Rong, Aiying & Lahdelma, Risto, 2017. "An efficient model and algorithm for the transmission-constrained multi-site combined heat and power system," European Journal of Operational Research, Elsevier, vol. 258(3), pages 1106-1117.
    7. Lehto, Jani & Oasmaa, Anja & Solantausta, Yrjö & Kytö, Matti & Chiaramonti, David, 2014. "Review of fuel oil quality and combustion of fast pyrolysis bio-oils from lignocellulosic biomass," Applied Energy, Elsevier, vol. 116(C), pages 178-190.
    8. Xu, Jing & Wang, Xiaoying & Gu, Yujiong & Ma, Suxia, 2023. "A data-based day-ahead scheduling optimization approach for regional integrated energy systems with varying operating conditions," Energy, Elsevier, vol. 283(C).
    9. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    10. Pérez-Iribarren, E. & González-Pino, I. & Azkorra-Larrinaga, Z. & Gómez-Arriarán, I., 2020. "Optimal design and operation of thermal energy storage systems in micro-cogeneration plants," Applied Energy, Elsevier, vol. 265(C).
    11. Bergthorson, Jeffrey M. & Thomson, Murray J., 2015. "A review of the combustion and emissions properties of advanced transportation biofuels and their impact on existing and future engines," Renewable and Sustainable Energy Reviews, Elsevier, vol. 42(C), pages 1393-1417.
    12. Hao Liang & Weihua Zhuang, 2014. "Stochastic Modeling and Optimization in a Microgrid: A Survey," Energies, MDPI, vol. 7(4), pages 1-24, March.
    13. Jean de Dieu Marcel Ufitikirezi & Martin Filip & Mohammad Ghorbani & Tomáš Zoubek & Pavel Olšan & Roman Bumbálek & Miroslav Strob & Petr Bartoš & Sandra Nicole Umurungi & Yves Theoneste Murindangabo &, 2024. "Agricultural Waste Valorization: Exploring Environmentally Friendly Approaches to Bioenergy Conversion," Sustainability, MDPI, vol. 16(9), pages 1-24, April.
    14. Ning, Jiajun & Xiong, Lixin, 2024. "Analysis of the dynamic evolution process of the digital transformation of renewable energy enterprises based on the cooperative and evolutionary game model," Energy, Elsevier, vol. 288(C).
    15. Dong, Zhen & Li, Zhongguo & Liang, Zhongchao & Xu, Yiqiao & Ding, Zhengtao, 2021. "Distributed neural network enhanced power generation strategy of large-scale wind power plant for power expansion," Applied Energy, Elsevier, vol. 303(C).
    16. Bhowmik, Chiranjib & Bhowmik, Sumit & Ray, Amitava & Pandey, Krishna Murari, 2017. "Optimal green energy planning for sustainable development: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 796-813.
    17. Zhu, Yilin & Xu, Yujie & Chen, Haisheng & Guo, Huan & Zhang, Hualiang & Zhou, Xuezhi & Shen, Haotian, 2023. "Optimal dispatch of a novel integrated energy system combined with multi-output organic Rankine cycle and hybrid energy storage," Applied Energy, Elsevier, vol. 343(C).
    18. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    19. Savelli, Iacopo & Morstyn, Thomas, 2021. "Electricity prices and tariffs to keep everyone happy: A framework for fixed and nodal prices coexistence in distribution grids with optimal tariffs for investment cost recovery," Omega, Elsevier, vol. 103(C).
    20. Liu, Shuwei & Tian, Jianyan & Ji, Zhengxiong & Dai, Yuanyuan & Guo, Hengkuan & Yang, Shengqiang, 2024. "Research on multi-digital twin and its application in wind power forecasting," Energy, Elsevier, vol. 292(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:376:y:2024:i:pa:s0306261924015629. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.