IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v338y2023ics0306261923003008.html
   My bibliography  Save this article

Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning

Author

Listed:
  • Zhuang, Dian
  • Gan, Vincent J.L.
  • Duygu Tekler, Zeynep
  • Chong, Adrian
  • Tian, Shuai
  • Shi, Xing

Abstract

Optimising HVAC operations towards human wellness and energy efficiency is a major challenge for smart facilities management, especially amid COVID situations. Although IoT sensors and deep learning were applied to support HVAC operations, the loss of forecasting accuracy in recursive prediction largely hinders their applications. This study presents a data-driven predictive control method with time-series forecasting (TSF) and reinforcement learning (RL), to examine various sensor metadata for HVAC system optimisation. This involves the development and validation of 16 Long Short-Term Memory (LSTM) based architectures with bi-directional processing, convolution, and attention mechanisms. The TSF models are comprehensively evaluated under independent, short-term recursive, and long-term recursive prediction scenarios. The optimal TSF models are integrated with a Soft Actor-Critic RL agent to analyse sensor metadata and optimise HVAC operations, achieving 17.4% energy savings and 16.9% thermal comfort improvement in the surrogate environment. The results show that recursive prediction leads to a significant reduction in model accuracy, and the effect is more pronounced in the temperature-humidity prediction model. The attention mechanism significantly improves prediction performance in both recursive and independent prediction scenarios. This study contributes new data-driven methods for smart HVAC operations in IoT-enabled intelligent buildings towards a human-centric built environment.

Suggested Citation

  • Zhuang, Dian & Gan, Vincent J.L. & Duygu Tekler, Zeynep & Chong, Adrian & Tian, Shuai & Shi, Xing, 2023. "Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning," Applied Energy, Elsevier, vol. 338(C).
    • Handle: RePEc:eee:appene:v:338:y:2023:i:c:s0306261923003008
    DOI: 10.1016/j.apenergy.2023.120936
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261923003008
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.120936?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yu, Xinran & Ergan, Semiha & Dedemen, Gokmen, 2019. "A data-driven approach to extract operational signatures of HVAC systems and analyze impact on electricity consumption," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    2. Blad, Christian & Bøgh, Simon & Kallesøe, Carsten Skovmose, 2022. "Data-driven Offline Reinforcement Learning for HVAC-systems," Energy, Elsevier, vol. 261(PB).
    3. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    4. Kim, Tae-Young & Cho, Sung-Bae, 2019. "Predicting residential energy consumption using CNN-LSTM neural networks," Energy, Elsevier, vol. 182(C), pages 72-81.
    5. Du, Yan & Zandi, Helia & Kotevska, Olivera & Kurte, Kuldeep & Munk, Jeffery & Amasyali, Kadir & Mckee, Evan & Li, Fangxing, 2021. "Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 281(C).
    6. Liu, Xiangfei & Ren, Mifeng & Yang, Zhile & Yan, Gaowei & Guo, Yuanjun & Cheng, Lan & Wu, Chengke, 2022. "A multi-step predictive deep reinforcement learning algorithm for HVAC control systems in smart buildings," Energy, Elsevier, vol. 259(C).
    7. Fan, Cheng & Wang, Jiayuan & Gang, Wenjie & Li, Shenghan, 2019. "Assessment of deep recurrent neural network-based strategies for short-term building energy predictions," Applied Energy, Elsevier, vol. 236(C), pages 700-710.
    8. Chung, Won Hee & Gu, Yeong Hyeon & Yoo, Seong Joon, 2022. "District heater load forecasting based on machine learning and parallel CNN-LSTM attention," Energy, Elsevier, vol. 246(C).
    9. Wang, Zhe & Hong, Tianzhen, 2020. "Reinforcement learning for building controls: The opportunities and challenges," Applied Energy, Elsevier, vol. 269(C).
    10. Blum, David & Wang, Zhe & Weyandt, Chris & Kim, Donghun & Wetter, Michael & Hong, Tianzhen & Piette, Mary Ann, 2022. "Field demonstration and implementation analysis of model predictive control in an office HVAC system," Applied Energy, Elsevier, vol. 318(C).
    11. Fazlipour, Zahra & Mashhour, Elaheh & Joorabian, Mahmood, 2022. "A deep model for short-term load forecasting applying a stacked autoencoder based on LSTM supported by a multi-stage attention mechanism," Applied Energy, Elsevier, vol. 327(C).
    12. Biemann, Marco & Scheller, Fabian & Liu, Xiufeng & Huang, Lizhen, 2021. "Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control," Applied Energy, Elsevier, vol. 298(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    2. Bian, Jianxiao & Wang, Jiarui & Yece, Qian, 2024. "A novel study on power consumption of an HVAC system using CatBoost and AdaBoost algorithms combined with the metaheuristic algorithms," Energy, Elsevier, vol. 302(C).
    3. Nik, Vahid M. & Hosseini, Mohammad, 2023. "CIRLEM: a synergic integration of Collective Intelligence and Reinforcement learning in Energy Management for enhanced climate resilience and lightweight computation," Applied Energy, Elsevier, vol. 350(C).
    4. Hu, Zehuan & Gao, Yuan & Sun, Luning & Mae, Masayuki & Imaizumi, Taiji, 2024. "Improved robust model predictive control for residential building air conditioning and photovoltaic power generation with battery energy storage system under weather forecast uncertainty," Applied Energy, Elsevier, vol. 371(C).
    5. Cui, Can & Xue, Jing, 2024. "Energy and comfort aware operation of multi-zone HVAC system through preference-inspired deep reinforcement learning," Energy, Elsevier, vol. 292(C).
    6. Yin, Cheng & Zeng, Xiangrui & Yin, Zhouping, 2024. "An improved data-driven predictive optimal control approach for designing hybrid electric vehicle energy management strategies," Applied Energy, Elsevier, vol. 375(C).
    7. Guo, Yuxiang & Qu, Shengli & Wang, Chuang & Xing, Ziwen & Duan, Kaiwen, 2024. "Optimal dynamic thermal management for data center via soft actor-critic algorithm with dynamic control interval and combined-value state space," Applied Energy, Elsevier, vol. 373(C).
    8. Loprete, Jason & Trojanowski, Rebecca & Butcher, Thomas & Longtin, Jon & Assanis, Dimitris, 2024. "Enabling residential heating decarbonization through hydronic low-temperature thermal distribution using forced-air assistive devices," Applied Energy, Elsevier, vol. 353(PA).
    9. Rosa Francesca De Masi & Nicoletta Del Regno & Antonio Gigante & Silvia Ruggiero & Alessandro Russo & Francesco Tariello & Giuseppe Peter Vanoli, 2023. "The Importance of Investing in the Energy Refurbishment of Hospitals: Results of a Case Study in a Mediterranean Climate," Sustainability, MDPI, vol. 15(14), pages 1-20, July.
    10. Luo, Jielin & Shen, Yongting & Yang, Hongxing, 2024. "Investigations on an integrated air-conditioning system using technologies of desiccant dehumidification, indirect evaporative cooling and CO2 capture," Applied Energy, Elsevier, vol. 369(C).
    11. Hashim Raza Khan & Wajahat Ahmed & Wasiq Masud & Urooj Alam & Kamran Arshad & Khaled Assaleh & Saad Ahmed Qazi, 2024. "Design and Experimental Results of an AIoT-Enabled, Energy-Efficient Ceiling Fan System," Sustainability, MDPI, vol. 16(12), pages 1-18, June.
    12. Troy Malatesta & Qilin Li & Jessica K. Breadsell & Christine Eon, 2023. "Distinguishing Household Groupings within a Precinct Based on Energy Usage Patterns Using Machine Learning Analysis," Energies, MDPI, vol. 16(10), pages 1-25, May.
    13. Zhou, Xinlei & Du, Han & Xue, Shan & Ma, Zhenjun, 2024. "Recent advances in data mining and machine learning for enhanced building energy management," Energy, Elsevier, vol. 307(C).
    14. Sheng Hu & Gongjin Yuan & Kaifeng Hu & Cong Liu & Minghu Wu, 2023. "Non-Intrusive Load Identification Method Based on KPCA-IGWO-RF," Energies, MDPI, vol. 16(12), pages 1-14, June.
    15. Aristeidis Mystakidis & Paraskevas Koukaras & Nikolaos Tsalikidis & Dimosthenis Ioannidis & Christos Tjortjis, 2024. "Energy Forecasting: A Comprehensive Review of Techniques and Technologies," Energies, MDPI, vol. 17(7), pages 1-33, March.
    16. Massimo Lauria & Maria Azzalin, 2024. "Digital Transformation in the Construction Sector: A Digital Twin for Seismic Safety in the Lifecycle of Buildings," Sustainability, MDPI, vol. 16(18), pages 1-22, September.
    17. Amjad Almusaed & Ibrahim Yitmen & Asaad Almssad, 2023. "Reviewing and Integrating AEC Practices into Industry 6.0: Strategies for Smart and Sustainable Future-Built Environments," Sustainability, MDPI, vol. 15(18), pages 1-27, September.
    18. Paraskevas Koukaras & Akeem Mustapha & Aristeidis Mystakidis & Christos Tjortjis, 2024. "Optimizing Building Short-Term Load Forecasting: A Comparative Analysis of Machine Learning Models," Energies, MDPI, vol. 17(6), pages 1-26, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Qin, Haosen & Meng, Tao & Chen, Kan & Li, Zhengwei, 2024. "A comparative study of DQN and D3QN for HVAC system optimization control," Energy, Elsevier, vol. 307(C).
    3. Blad, C. & Bøgh, S. & Kallesøe, C. & Raftery, Paul, 2023. "A laboratory test of an Offline-trained Multi-Agent Reinforcement Learning Algorithm for Heating Systems," Applied Energy, Elsevier, vol. 337(C).
    4. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    5. Pinto, Giuseppe & Kathirgamanathan, Anjukan & Mangina, Eleni & Finn, Donal P. & Capozzoli, Alfonso, 2022. "Enhancing energy management in grid-interactive buildings: A comparison among cooperative and coordinated architectures," Applied Energy, Elsevier, vol. 310(C).
    6. Guo, Yuxiang & Qu, Shengli & Wang, Chuang & Xing, Ziwen & Duan, Kaiwen, 2024. "Optimal dynamic thermal management for data center via soft actor-critic algorithm with dynamic control interval and combined-value state space," Applied Energy, Elsevier, vol. 373(C).
    7. Wang, Xuezheng & Dong, Bing, 2024. "Long-term experimental evaluation and comparison of advanced controls for HVAC systems," Applied Energy, Elsevier, vol. 371(C).
    8. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    9. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    10. Huakun Huang & Dingrong Dai & Longtao Guo & Sihui Xue & Huijun Wu, 2023. "AI and Big Data-Empowered Low-Carbon Buildings: Challenges and Prospects," Sustainability, MDPI, vol. 15(16), pages 1-21, August.
    11. Song, Yuguang & Xia, Mingchao & Chen, Qifang & Chen, Fangjian, 2023. "A data-model fusion dispatch strategy for the building energy flexibility based on the digital twin," Applied Energy, Elsevier, vol. 332(C).
    12. Liu, Shuo & Liu, Xiaohua & Zhang, Tao & Wang, Chaoliang & Liu, Wei, 2024. "Joint optimization for temperature and humidity independent control system based on multi-agent reinforcement learning with cooperative mechanisms," Applied Energy, Elsevier, vol. 375(C).
    13. Zhou, Xinlei & Du, Han & Xue, Shan & Ma, Zhenjun, 2024. "Recent advances in data mining and machine learning for enhanced building energy management," Energy, Elsevier, vol. 307(C).
    14. Ayas Shaqour & Aya Hagishima, 2022. "Systematic Review on Deep Reinforcement Learning-Based Energy Management for Different Building Types," Energies, MDPI, vol. 15(22), pages 1-27, November.
    15. Gao, Yuan & Matsunami, Yuki & Miyata, Shohei & Akashi, Yasunori, 2022. "Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system," Applied Energy, Elsevier, vol. 326(C).
    16. Coraci, Davide & Brandi, Silvio & Hong, Tianzhen & Capozzoli, Alfonso, 2023. "Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings," Applied Energy, Elsevier, vol. 333(C).
    17. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    18. Silvestri, Alberto & Coraci, Davide & Brandi, Silvio & Capozzoli, Alfonso & Borkowski, Esther & Köhler, Johannes & Wu, Duan & Zeilinger, Melanie N. & Schlueter, Arno, 2024. "Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control," Applied Energy, Elsevier, vol. 368(C).
    19. Chen, Minghao & Xie, Zhiyuan & Sun, Yi & Zheng, Shunlin, 2023. "The predictive management in campus heating system based on deep reinforcement learning and probabilistic heat demands forecasting," Applied Energy, Elsevier, vol. 350(C).
    20. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:338:y:2023:i:c:s0306261923003008. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.