IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v356y2024ics030626192301721x.html
   My bibliography  Save this article

Deep clustering of reinforcement learning based on the bang-bang principle to optimize the energy in multi-boiler for intelligent buildings

Author

Listed:
  • Homod, Raad Z.
  • Munahi, Basil Sh.
  • Mohammed, Hayder Ibrahim
  • Albadr, Musatafa Abbas Abbood
  • Abderrahmane, AISSA
  • Mahdi, Jasim M.
  • Ben Hamida, Mohamed Bechir
  • Alhasnawi, Bilal Naji
  • Albahri, A.S.
  • Togun, Hussein
  • Alqsair, Umar F.
  • Yaseen, Zaher Mundher

Abstract

The bang-bang relays of the multiple-boiler system (MBS) control, are characterized by complex limiter saturation functions and classified as fixed parameters. Their action signals cannot precisely control the nonlinear dynamic building heating demand over their entire range of operation. Moreover, in a mono-boiler system, the bang-bang controller endures increasing short cycling over partial load time due to the heating system being considered to have an oversized boiler at most times of running, thus promoting high energy consumption and fluctuating indoor thermal comfort. So, it is difficult to cope with uncertainties in outdoor environments and indoor heating load. Hence, this study formulates the MBS control problem as a dynamic Markov decision process and applies a deep clustering of reinforcement learning approach to obtain the optimal control policy through interaction with the environment based on multi-agent learning according to bang-bang action. With such an approach, adopting a new boiler sequencing control (BSC) strategy using deep clustering of reinforcement learning based on a bang-bang (DCRLBB) manner. The deep clustering is configured to break Lagrangian trajectory curves into piecewise segments to represent the RL agent's action policy. The agent's action policy signals are configured from the bang-bang reward formula based on trade-off implications to be more adjustable than traditional fixed parameters such as fuzzy bang-bang controller (FBBC). The agent of BSC significantly affects the energy performance of the MBS, whereas the other agent resizes boiler capacity by acting to adjust the boiler solenoid fuel valve. The comparison of results between the proposed strategy and conventional FBBC shows distinct differences in the superior response of DCRLBB under dynamic indoor/outdoor actual conditions and energy saving by more than 32% while maintaining the indoor thermal in the comfortable range.

Suggested Citation

  • Homod, Raad Z. & Munahi, Basil Sh. & Mohammed, Hayder Ibrahim & Albadr, Musatafa Abbas Abbood & Abderrahmane, AISSA & Mahdi, Jasim M. & Ben Hamida, Mohamed Bechir & Alhasnawi, Bilal Naji & Albahri, A., 2024. "Deep clustering of reinforcement learning based on the bang-bang principle to optimize the energy in multi-boiler for intelligent buildings," Applied Energy, Elsevier, vol. 356(C).
    • Handle: RePEc:eee:appene:v:356:y:2024:i:c:s030626192301721x
    DOI: 10.1016/j.apenergy.2023.122357
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030626192301721X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.122357?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Homod, Raad Z., 2014. "Assessment regarding energy saving and decoupling for different AHU (air handling unit) and control strategies in the hot-humid climatic region of Iraq," Energy, Elsevier, vol. 74(C), pages 762-774.
    2. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    3. Fujishima, Hidekatsu & Takekoshi, Kenichi & Kuroki, Tomoyuki & Tanaka, Atsushi & Otsuka, Keiichi & Okubo, Masaaki, 2013. "Towards ideal NOx control technology for bio-oils and a gas multi-fuel boiler system using a plasma-chemical hybrid process," Applied Energy, Elsevier, vol. 111(C), pages 394-400.
    4. Maytham S. Ahmed & Azah Mohamed & Raad Z. Homod & Hussain Shareef, 2016. "Hybrid LSA-ANN Based Home Energy Management Scheduling Controller for Residential Demand Response Strategy," Energies, MDPI, vol. 9(9), pages 1-20, September.
    5. Wang, Kang & Wang, Haixin & Yang, Zihao & Feng, Jiawei & Li, Yanzhen & Yang, Junyou & Chen, Zhe, 2023. "A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 343(C).
    6. Homod, Raad Z. & Gaeid, Khalaf S. & Dawood, Suroor M. & Hatami, Alireza & Sahari, Khairul S., 2020. "Evaluation of energy-saving potential for optimal time response of HVAC control system in smart buildings," Applied Energy, Elsevier, vol. 271(C).
    7. Homod, Raad Z. & Sahari, Khairul Salleh Mohamed & Almurib, Haider A.F., 2014. "Energy saving by integrated control of natural ventilation and HVAC systems using model guide for comparison," Renewable Energy, Elsevier, vol. 71(C), pages 639-650.
    8. Liao, Weicheng & Zhang, Xiaoyue & Li, Zhen, 2022. "Experimental investigation on the performance of a boiler system with flue gas dehumidification and combustion air humidification," Applied Energy, Elsevier, vol. 323(C).
    9. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    10. Homod, Raad Z. & Togun, Hussein & Ateeq, Adnan A. & Al-Mousawi, Fadhel Noraldeen & Yaseen, Zaher Mundher & Al-Kouz, Wael & Hussein, Ahmed Kadhim & Alawi, Omer A. & Goodarzi, Marjan & Ahmadi, Goodarz, 2022. "An innovative clustering technique to generate hybrid modeling of cooling coils for energy analysis: A case study for control performance in HVAC systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 166(C).
    11. Zheng, Peijun & Zhou, Heng & Liu, Jiang & Nakanishi, Yosuke, 2023. "Interpretable building energy consumption forecasting using spectral clustering algorithm and temporal fusion transformers architecture," Applied Energy, Elsevier, vol. 349(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liu, Yuntao & Cui, Can, 2024. "A bi-level real-time optimal control strategy for thermal coupled multi-zone dedicated outside air system-assisted HVAC systems," Energy, Elsevier, vol. 306(C).
    2. Liu, Shuo & Liu, Xiaohua & Zhang, Tao & Wang, Chaoliang & Liu, Wei, 2024. "Joint optimization for temperature and humidity independent control system based on multi-agent reinforcement learning with cooperative mechanisms," Applied Energy, Elsevier, vol. 375(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Homod, Raad Z. & Togun, Hussein & Ateeq, Adnan A. & Al-Mousawi, Fadhel Noraldeen & Yaseen, Zaher Mundher & Al-Kouz, Wael & Hussein, Ahmed Kadhim & Alawi, Omer A. & Goodarzi, Marjan & Ahmadi, Goodarz, 2022. "An innovative clustering technique to generate hybrid modeling of cooling coils for energy analysis: A case study for control performance in HVAC systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 166(C).
    2. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    3. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    4. Homod, Raad Z., 2018. "Analysis and optimization of HVAC control systems based on energy and performance considerations for smart buildings," Renewable Energy, Elsevier, vol. 126(C), pages 49-64.
    5. Waseem, Muhammad & Lin, Zhenzhi & Liu, Shengyuan & Zhang, Zhi & Aziz, Tarique & Khan, Danish, 2021. "Fuzzy compromised solution-based novel home appliances scheduling and demand response with optimal dispatch of distributed energy resources," Applied Energy, Elsevier, vol. 290(C).
    6. Homod, Raad Z. & Gaeid, Khalaf S. & Dawood, Suroor M. & Hatami, Alireza & Sahari, Khairul S., 2020. "Evaluation of energy-saving potential for optimal time response of HVAC control system in smart buildings," Applied Energy, Elsevier, vol. 271(C).
    7. Lei, Yue & Zhan, Sicheng & Ono, Eikichi & Peng, Yuzhen & Zhang, Zhiang & Hasama, Takamasa & Chong, Adrian, 2022. "A practical deep reinforcement learning framework for multivariate occupant-centric control in buildings," Applied Energy, Elsevier, vol. 324(C).
    8. Alexandre Correia & Luís Miguel Ferreira & Paulo Coimbra & Pedro Moura & Aníbal T. de Almeida, 2022. "Smart Thermostats for a Campus Microgrid: Demand Control and Improving Air Quality," Energies, MDPI, vol. 15(4), pages 1-21, February.
    9. Kritana Prueksakorn & Cheng-Xu Piao & Hyunchul Ha & Taehyeung Kim, 2015. "Computational and Experimental Investigation for an Optimal Design of Industrial Windows to Allow Natural Ventilation during Wind-Driven Rain," Sustainability, MDPI, vol. 7(8), pages 1-22, August.
    10. Giovanni Pau & Mario Collotta & Antonio Ruano & Jiahu Qin, 2017. "Smart Home Energy Management," Energies, MDPI, vol. 10(3), pages 1-5, March.
    11. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    12. Zhuang, Chaoqun & Wang, Shengwei & Shan, Kui, 2019. "Adaptive full-range decoupled ventilation strategy and air-conditioning systems for cleanrooms and buildings requiring strict humidity control and their performance evaluation," Energy, Elsevier, vol. 168(C), pages 883-896.
    13. Zhang, Tianren & Huang, Yuping & Liao, Hui & Liang, Yu, 2023. "A hybrid electric vehicle load classification and forecasting approach based on GBDT algorithm and temporal convolutional network," Applied Energy, Elsevier, vol. 351(C).
    14. He, Yueer & Liu, Meng & Kvan, Thomas & Peng, Shini, 2017. "An enthalpy-based energy savings estimation method targeting thermal comfort level in naturally ventilated buildings in hot-humid summer zones," Applied Energy, Elsevier, vol. 187(C), pages 717-731.
    15. Ussama Assad & Muhammad Arshad Shehzad Hassan & Umar Farooq & Asif Kabir & Muhammad Zeeshan Khan & S. Sabahat H. Bukhari & Zain ul Abidin Jaffri & Judit Oláh & József Popp, 2022. "Smart Grid, Demand Response and Optimization: A Critical Review of Computational Methods," Energies, MDPI, vol. 15(6), pages 1-36, March.
    16. Yuchun Li & Yinghua Han & Jinkuan Wang & Qiang Zhao, 2018. "A MBCRF Algorithm Based on Ensemble Learning for Building Demand Response Considering the Thermal Comfort," Energies, MDPI, vol. 11(12), pages 1-20, December.
    17. Adnan Ahmad & Asif Khan & Nadeem Javaid & Hafiz Majid Hussain & Wadood Abdul & Ahmad Almogren & Atif Alamri & Iftikhar Azim Niaz, 2017. "An Optimized Home Energy Management System with Integrated Renewable Energy and Storage Resources," Energies, MDPI, vol. 10(4), pages 1-35, April.
    18. Jia Ning & Yi Tang & Qian Chen & Jianming Wang & Jianhua Zhou & Bingtuan Gao, 2017. "A Bi-Level Coordinated Optimization Strategy for Smart Appliances Considering Online Demand Response Potential," Energies, MDPI, vol. 10(4), pages 1-16, April.
    19. Liu, Yiren & Zhao, Xiangyu & Qin, S. Joe, 2024. "Dynamically engineered multi-modal feature learning for predictions of office building cooling loads," Applied Energy, Elsevier, vol. 355(C).
    20. Lee, Minjung & Ham, Jeonggyun & Lee, Jeong-Won & Cho, Honghyun, 2023. "Analysis of thermal comfort, energy consumption, and CO2 reduction of indoor space according to the type of local heating under winter rest conditions," Energy, Elsevier, vol. 268(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:356:y:2024:i:c:s030626192301721x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.