IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v378y2025ipas0306261924021639.html
   My bibliography  Save this article

Resilient energy management of a multi-energy building under low-temperature district heating: A deep reinforcement learning approach

Author

Listed:
  • Wang, Jiawei
  • Wang, Yi
  • Qiu, Dawei
  • Su, Hanguang
  • Strbac, Goran
  • Gao, Zhiwei

Abstract

The corrective control of a building-level multi-energy system (MES) for emergency load shedding is essential to optimize the operating cost after contingency. For a Danish case, the heating devices in the building are connected to a developing low-temperature district heating (LTDH) system and operated under a heat market. Due to the coupling between the electrical power and heating system, an electricity outage can be propagated to the heating network, and heat prices as well as tariffs can impact the MES operating cost. In the previous studies, only electrical load shedding is modeled, while the impact of electricity outages on heating system operation and heat load control is ignored. On the other hand, the problem is traditionally solved by model-based optimization methods which are highly nonconvex leading to high computing complexity. Moreover, operating uncertainties can lead to infeasible solutions. To address these challenges, this paper proposes a deep reinforcement learning-based corrective control method for the resilient energy management of a building-level MES. In the method, the proximal policy optimization algorithm is applied, where multiple uncertainties, system dynamics, and operating constraints are considered. A case study of a real-life residential building connected to the LTDH system in Denmark is carried out, where electricity outages are simulated. The results verify the performance of the proposed method in achieving resilient energy management of the MES.

Suggested Citation

  • Wang, Jiawei & Wang, Yi & Qiu, Dawei & Su, Hanguang & Strbac, Goran & Gao, Zhiwei, 2025. "Resilient energy management of a multi-energy building under low-temperature district heating: A deep reinforcement learning approach," Applied Energy, Elsevier, vol. 378(PA).
    • Handle: RePEc:eee:appene:v:378:y:2025:i:pa:s0306261924021639
    DOI: 10.1016/j.apenergy.2024.124780
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924021639
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.124780?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Østergaard, Dorte Skaarup & Svendsen, Svend, 2016. "Replacing critical radiators to increase the potential to use low-temperature district heating – A case study of 4 Danish single-family houses from the 1930s," Energy, Elsevier, vol. 110(C), pages 75-84.
    2. Mikielewicz, Jarosław & Mikielewicz, Dariusz, 2023. "Comparison of traditional district heating with low temperature district heating systems featuring organic Rankine cycle and heat pump," Energy, Elsevier, vol. 281(C).
    3. Yang, Xiaochen & Li, Hongwei & Svendsen, Svend, 2016. "Evaluations of different domestic hot water preparing methods with ultra-low-temperature district heating," Energy, Elsevier, vol. 109(C), pages 248-259.
    4. Ye, Jin & Wang, Xianlian & Hua, Qingsong & Sun, Li, 2024. "Deep reinforcement learning based energy management of a hybrid electricity-heat-hydrogen energy system with demand response," Energy, Elsevier, vol. 305(C).
    5. Gadd, Henrik & Werner, Sven, 2013. "Daily heat load variations in Swedish district heating systems," Applied Energy, Elsevier, vol. 106(C), pages 47-55.
    6. Yu, Min Gyung & Pavlak, Gregory S., 2023. "Risk-aware sizing and transactive control of building portfolios with thermal energy storage," Applied Energy, Elsevier, vol. 332(C).
    7. Sun, Qirun & Wu, Zhi & Ma, Zhoujun & Gu, Wei & Zhang, Xiao-Ping & Lu, Yuping & Liu, Pengxiang, 2022. "Resilience enhancement strategy for multi-energy systems considering multi-stage recovery process and multi-energy coordination," Energy, Elsevier, vol. 241(C).
    8. Zhao, Liyuan & Yang, Ting & Li, Wei & Zomaya, Albert Y., 2022. "Deep reinforcement learning-based joint load scheduling for household multi-energy system," Applied Energy, Elsevier, vol. 324(C).
    9. Schmitz, William Ismael & Schmitz, Magdiel & Canha, Luciane Neves & Garcia, Vinícius Jacques, 2020. "Proactive home energy storage management system to severe weather scenarios," Applied Energy, Elsevier, vol. 279(C).
    10. Mehrjerdi, Hasan & Hemmati, Reza, 2020. "Coordination of vehicle-to-home and renewable capacity resources for energy management in resilience and self-healing building," Renewable Energy, Elsevier, vol. 146(C), pages 568-579.
    11. Wang, Jiawei & You, Shi & Zong, Yi & Cai, Hanmin & Træholt, Chresten & Dong, Zhao Yang, 2019. "Investigation of real-time flexibility of combined heat and power plants in district heating applications," Applied Energy, Elsevier, vol. 237(C), pages 196-209.
    12. Guo, Chenyu & Wang, Xin & Zheng, Yihui & Zhang, Feng, 2022. "Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    13. Mansour-lakouraj, Mohammad & Shahabi, Majid, 2019. "Comprehensive analysis of risk-based energy management for dependent micro-grid under normal and emergency operations," Energy, Elsevier, vol. 171(C), pages 928-943.
    14. Thomas, Dimitrios & Deblecker, Olivier & Ioakimidis, Christos S., 2018. "Optimal operation of an energy management system for a grid-connected smart building considering photovoltaics’ uncertainty and stochastic electric vehicles’ driving schedule," Applied Energy, Elsevier, vol. 210(C), pages 1188-1206.
    15. Hafez, A.M. & Kassem, M.A. & Huzayyin, O.A., 2018. "Smart adaptive model for dynamic simulation of horizontal thermally stratified storage tanks," Energy, Elsevier, vol. 142(C), pages 782-792.
    16. Mishra, Dillip Kumar & Ghadi, Mojtaba Jabbari & Li, Li & Zhang, Jiangfeng & Hossain, M.J., 2022. "Active distribution system resilience quantification and enhancement through multi-microgrid and mobile energy storage," Applied Energy, Elsevier, vol. 311(C).
    17. Ghilardi, Lavinia Marina Paola & Castelli, Alessandro Francesco & Moretti, Luca & Morini, Mirko & Martelli, Emanuele, 2021. "Co-optimization of multi-energy system operation, district heating/cooling network and thermal comfort management for buildings," Applied Energy, Elsevier, vol. 302(C).
    18. Yokoyama, Ryohei & Kitano, Hiroyuki & Wakui, Tetsuya, 2017. "Optimal operation of heat supply systems with piping network," Energy, Elsevier, vol. 137(C), pages 888-897.
    19. Pickering, Bryn & Choudhary, Ruchi, 2021. "Quantifying resilience in energy systems with out-of-sample testing," Applied Energy, Elsevier, vol. 285(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Østergaard, Dorte Skaarup & Smith, Kevin Michael & Tunzi, Michele & Svendsen, Svend, 2022. "Low-temperature operation of heating systems to enable 4th generation district heating: A review," Energy, Elsevier, vol. 248(C).
    2. Li, Haoran & Hou, Juan & Hong, Tianzhen & Nord, Natasa, 2022. "Distinguish between the economic optimal and lowest distribution temperatures for heat-prosumer-based district heating systems with short-term thermal energy storage," Energy, Elsevier, vol. 248(C).
    3. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    4. Lv, Hang & Wu, Qiong & Ren, Hongbo & Li, Qifen & Zhou, Weisheng, 2024. "A two-stage decision-making approach for optimal design and operation of integrated energy systems considering multiple uncertainties and diverse resilience needs," Energy, Elsevier, vol. 305(C).
    5. Danica Djurić Ilić, 2020. "Classification of Measures for Dealing with District Heating Load Variations—A Systematic Review," Energies, MDPI, vol. 14(1), pages 1-27, December.
    6. Mohseni, Soheil & Brent, Alan C. & Kelly, Scott & Browne, Will N., 2022. "Demand response-integrated investment and operational planning of renewable and sustainable energy systems considering forecast uncertainties: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 158(C).
    7. Alessandro Guzzini & Marco Pellegrini & Edoardo Pelliconi & Cesare Saccani, 2020. "Low Temperature District Heating: An Expert Opinion Survey," Energies, MDPI, vol. 13(4), pages 1-34, February.
    8. Buonomano, Annamaria, 2020. "Building to Vehicle to Building concept: A comprehensive parametric and sensitivity analysis for decision making aims," Applied Energy, Elsevier, vol. 261(C).
    9. Wang, Can & Zhang, Jiaheng & Wang, Aoqi & Wang, Zhen & Yang, Nan & Zhao, Zhuoli & Lai, Chun Sing & Lai, Loi Lei, 2024. "Prioritized sum-tree experience replay TD3 DRL-based online energy management of a residential microgrid," Applied Energy, Elsevier, vol. 368(C).
    10. Hakan İbrahim Tol & Habtamu Bayera Madessa, 2024. "Return-Temperature Reduction at District Heating Systems: Focus on End-User Sites," Energies, MDPI, vol. 17(19), pages 1-46, September.
    11. Thorsen, Jan Eric & Gudmundsson, Oddgeir & Tunzi, Michele & Esbensen, Torben, 2024. "Aftercooling concept: An innovative substation ready for 4th generation district heating networks," Energy, Elsevier, vol. 293(C).
    12. Lund, Henrik & Østergaard, Poul Alberg & Chang, Miguel & Werner, Sven & Svendsen, Svend & Sorknæs, Peter & Thorsen, Jan Eric & Hvelplund, Frede & Mortensen, Bent Ole Gram & Mathiesen, Brian Vad & Boje, 2018. "The status of 4th generation district heating: Research and results," Energy, Elsevier, vol. 164(C), pages 147-159.
    13. Sridharan, S. & Sivakumar, S. & Shanmugasundaram, N. & Swapna, S. & Vasan Prabhu, V., 2023. "A hybrid approach based energy management for building resilience against power outage by shared parking station for EVs," Renewable Energy, Elsevier, vol. 216(C).
    14. Isaías Gomes & Karol Bot & Maria Graça Ruano & António Ruano, 2022. "Recent Techniques Used in Home Energy Management Systems: A Review," Energies, MDPI, vol. 15(8), pages 1-41, April.
    15. Volkova, Anna & Mašatin, Vladislav & Siirde, Andres, 2018. "Methodology for evaluating the transition process dynamics towards 4th generation district heating networks," Energy, Elsevier, vol. 150(C), pages 253-261.
    16. Barone, Giovanni & Buonomano, Annamaria & Forzano, Cesare & Giuzio, Giovanni Francesco & Palombo, Adolfo, 2020. "Increasing self-consumption of renewable energy through the Building to Vehicle to Building approach applied to multiple users connected in a virtual micro-grid," Renewable Energy, Elsevier, vol. 159(C), pages 1165-1176.
    17. Averfalk, Helge & Werner, Sven, 2018. "Novel low temperature heat distribution technology," Energy, Elsevier, vol. 145(C), pages 526-539.
    18. Tian, Man-Wen & Talebizadehsardari, Pouyan, 2021. "Energy cost and efficiency analysis of building resilience against power outage by shared parking station for electric vehicles and demand response program," Energy, Elsevier, vol. 215(PB).
    19. Li, Linyue & Li, Chenxiao & Alharthi, Yahya Z. & Wang, Yubin & Safaraliev, Murodbek, 2025. "A two-layer economic resilience model for distribution network restoration after natural disasters," Applied Energy, Elsevier, vol. 377(PC).
    20. Guelpa, Elisa, 2021. "Impact of thermal masses on the peak load in district heating systems," Energy, Elsevier, vol. 214(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:378:y:2025:i:pa:s0306261924021639. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.