IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v375y2024ics0306261924013515.html
   My bibliography  Save this article

Joint optimization for temperature and humidity independent control system based on multi-agent reinforcement learning with cooperative mechanisms

Author

Listed:
  • Liu, Shuo
  • Liu, Xiaohua
  • Zhang, Tao
  • Wang, Chaoliang
  • Liu, Wei

Abstract

The coupling of indoor thermal environment parameters and the complexity of subsystem control pose challenges to further enhancing the energy efficiency of air conditioning systems. This study adopts a multi-agent deep reinforcement learning algorithm for optimal control of a temperature and humidity independent control system combining radiant ceiling cooling with desiccant dehumidification. In the co-simulation process, Python completes the algorithm construction and employs the system and building models constructed by EnergyPlus to achieve the training of agents. This study aims to analyse the optimisation effects of different algorithms, discuss the behavioural characteristics among multiple agents during the training process, and reveal the impact of humidity intelligent control on the energy saving and energy-use flexibility effects. Compared to conventional single-agent optimization control methods, this study integrates interdependent control actions and leverages a cooperative mechanism to enhance system optimization. Simulation results demonstrate that the method can avoid the local cognitive limitations of individual agents in dynamic environments, achieving multi-objective optimization. Notably, the humidity control agent autonomously reduces humidity to assist in lowering the supply water temperature, thereby managing high sensible heat loads while avoiding condensation risks of the radiant cooling panels. The optimized strategy reduces chiller energy consumption by 41.4%, with 16% of this reduction attributed to humidity control. Furthermore, humidity control enables the chiller to operate at varying power levels, enhancing the energy-use flexibility. This study provides a replicable and transferable solution for multi-subsystem optimal control.

Suggested Citation

  • Liu, Shuo & Liu, Xiaohua & Zhang, Tao & Wang, Chaoliang & Liu, Wei, 2024. "Joint optimization for temperature and humidity independent control system based on multi-agent reinforcement learning with cooperative mechanisms," Applied Energy, Elsevier, vol. 375(C).
    • Handle: RePEc:eee:appene:v:375:y:2024:i:c:s0306261924013515
    DOI: 10.1016/j.apenergy.2024.123968
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924013515
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.123968?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    2. Liu, Shuo & Jang, Hyusan & Yeo, Myoung-Souk, 2023. "Experimental study on the operating characteristic of the desiccant cooling systems with the potential of condensing heat recovery," Energy, Elsevier, vol. 283(C).
    3. Wang, Hao & Chen, Xiwen & Vital, Natan & Duffy, Edward & Razi, Abolfazl, 2024. "Energy optimization for HVAC systems in multi-VAV open offices: A deep reinforcement learning approach," Applied Energy, Elsevier, vol. 356(C).
    4. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    5. Du, Yan & Zandi, Helia & Kotevska, Olivera & Kurte, Kuldeep & Munk, Jeffery & Amasyali, Kadir & Mckee, Evan & Li, Fangxing, 2021. "Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 281(C).
    6. Homod, Raad Z. & Munahi, Basil Sh. & Mohammed, Hayder Ibrahim & Albadr, Musatafa Abbas Abbood & Abderrahmane, AISSA & Mahdi, Jasim M. & Ben Hamida, Mohamed Bechir & Alhasnawi, Bilal Naji & Albahri, A., 2024. "Deep clustering of reinforcement learning based on the bang-bang principle to optimize the energy in multi-boiler for intelligent buildings," Applied Energy, Elsevier, vol. 356(C).
    7. Xie, Yiwei & Hu, Pingfang & Zhu, Na & Lei, Fei & Xing, Lu & Xu, Linghong, 2020. "Collaborative optimization of ground source heat pump-radiant ceiling air conditioning system based on response surface method and NSGA-II," Renewable Energy, Elsevier, vol. 147(P1), pages 249-264.
    8. Shen, Rendong & Zhong, Shengyuan & Wen, Xin & An, Qingsong & Zheng, Ruifan & Li, Yang & Zhao, Jun, 2022. "Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy," Applied Energy, Elsevier, vol. 312(C).
    9. Wang, Zhe & Hong, Tianzhen, 2020. "Reinforcement learning for building controls: The opportunities and challenges," Applied Energy, Elsevier, vol. 269(C).
    10. Blum, David & Wang, Zhe & Weyandt, Chris & Kim, Donghun & Wetter, Michael & Hong, Tianzhen & Piette, Mary Ann, 2022. "Field demonstration and implementation analysis of model predictive control in an office HVAC system," Applied Energy, Elsevier, vol. 318(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiang, Yuliang & Zhu, Shanying & Xu, Qimin & Yang, Bo & Guan, Xinping, 2023. "Hybrid modeling-based temperature and humidity adaptive control for a multi-zone HVAC system," Applied Energy, Elsevier, vol. 334(C).
    2. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    3. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    4. Biemann, Marco & Scheller, Fabian & Liu, Xiufeng & Huang, Lizhen, 2021. "Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control," Applied Energy, Elsevier, vol. 298(C).
    5. Zhuang, Dian & Gan, Vincent J.L. & Duygu Tekler, Zeynep & Chong, Adrian & Tian, Shuai & Shi, Xing, 2023. "Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning," Applied Energy, Elsevier, vol. 338(C).
    6. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    7. Liu, Mingzhe & Guo, Mingyue & Fu, Yangyang & O’Neill, Zheng & Gao, Yuan, 2024. "Expert-guided imitation learning for energy management: Evaluating GAIL’s performance in building control applications," Applied Energy, Elsevier, vol. 372(C).
    8. Langer, Lissy & Volling, Thomas, 2022. "A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems," Applied Energy, Elsevier, vol. 327(C).
    9. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    10. Zhang, Bin & Hu, Weihao & Ghias, Amer M.Y.M. & Xu, Xiao & Chen, Zhe, 2022. "Multi-agent deep reinforcement learning-based coordination control for grid-aware multi-buildings," Applied Energy, Elsevier, vol. 328(C).
    11. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    12. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    13. Pinto, Giuseppe & Kathirgamanathan, Anjukan & Mangina, Eleni & Finn, Donal P. & Capozzoli, Alfonso, 2022. "Enhancing energy management in grid-interactive buildings: A comparison among cooperative and coordinated architectures," Applied Energy, Elsevier, vol. 310(C).
    14. Guo, Yuxiang & Qu, Shengli & Wang, Chuang & Xing, Ziwen & Duan, Kaiwen, 2024. "Optimal dynamic thermal management for data center via soft actor-critic algorithm with dynamic control interval and combined-value state space," Applied Energy, Elsevier, vol. 373(C).
    15. Wang, Xuezheng & Dong, Bing, 2024. "Long-term experimental evaluation and comparison of advanced controls for HVAC systems," Applied Energy, Elsevier, vol. 371(C).
    16. Hindreen Rashid Abdulqadir & Adnan Mohsin Abdulazeez, 2021. "Reinforcement Learning and Modeling Techniques: A Review," International Journal of Science and Business, IJSAB International, vol. 5(3), pages 174-189.
    17. Touzani, Samir & Prakash, Anand Krishnan & Wang, Zhe & Agarwal, Shreya & Pritoni, Marco & Kiran, Mariam & Brown, Richard & Granderson, Jessica, 2021. "Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency," Applied Energy, Elsevier, vol. 304(C).
    18. Hernandez-Matheus, Alejandro & Löschenbrand, Markus & Berg, Kjersti & Fuchs, Ida & Aragüés-Peñalba, Mònica & Bullich-Massagué, Eduard & Sumper, Andreas, 2022. "A systematic review of machine learning techniques related to local energy communities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    19. Zheng, Lingwei & Wu, Hao & Guo, Siqi & Sun, Xinyu, 2023. "Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy," Energy, Elsevier, vol. 277(C).
    20. Davide Deltetto & Davide Coraci & Giuseppe Pinto & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Exploring the Potentialities of Deep Reinforcement Learning for Incentive-Based Demand Response in a Cluster of Small Commercial Buildings," Energies, MDPI, vol. 14(10), pages 1-25, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:375:y:2024:i:c:s0306261924013515. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.