IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i24p8210-d697081.html
   My bibliography  Save this article

Energy Management Model for HVAC Control Supported by Reinforcement Learning

Author

Listed:
  • Pedro Macieira

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

  • Luis Gomes

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

  • Zita Vale

    (GECAD—Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO), P-4200-072 Porto, Portugal)

Abstract

Heating, ventilating, and air conditioning (HVAC) units account for a significant consumption share in buildings, namely office buildings. Therefore, this paper addresses the possibility of having an intelligent and more cost-effective solution for the management of HVAC units in office buildings. The method applied in this paper divides the addressed problem into three steps: (i) the continuous acquisition of data provided by an open-source building energy management systems, (ii) the proposed learning and predictive model able to predict if users will be working in a given location, and (iii) the proposed decision model to manage the HVAC units according to the prediction of users, current environmental context, and current energy prices. The results show that the proposed predictive model was able to achieve a 93.8% accuracy and that the proposed decision tree enabled the maintenance of users’ comfort. The results demonstrate that the proposed solution is able to run in real-time in a real office building, making it a possible solution for smart buildings.

Suggested Citation

  • Pedro Macieira & Luis Gomes & Zita Vale, 2021. "Energy Management Model for HVAC Control Supported by Reinforcement Learning," Energies, MDPI, vol. 14(24), pages 1-14, December.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:24:p:8210-:d:697081
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/24/8210/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/24/8210/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Robert Lou & Kevin P. Hallinan & Kefan Huang & Timothy Reissman, 2020. "Smart Wifi Thermostat-Enabled Thermal Comfort Control in Residences," Sustainability, MDPI, vol. 12(5), pages 1-15, March.
    2. Dana-Mihaela Petroșanu & George Căruțașu & Nicoleta Luminița Căruțașu & Alexandru Pîrjan, 2019. "A Review of the Recent Developments in Integrating Machine Learning Models with Sensor Devices in the Smart Buildings Sector with a View to Attaining Enhanced Sensing, Energy Efficiency, and Optimal B," Energies, MDPI, vol. 12(24), pages 1-64, December.
    3. Jung, Wooyoung & Jazizadeh, Farrokh, 2019. "Human-in-the-loop HVAC operations: A quantitative review on occupancy, comfort, and energy-efficiency dimensions," Applied Energy, Elsevier, vol. 239(C), pages 1471-1508.
    4. Adhikari, Rajendra & Pipattanasomporn, M. & Rahman, S., 2018. "An algorithm for optimal management of aggregated HVAC power demand using smart thermostats," Applied Energy, Elsevier, vol. 217(C), pages 166-177.
    5. Ingo Liere-Netheler & Frank Schuldt & Karsten von Maydell & Carsten Agert, 2020. "Simulation of Incidental Distributed Generation Curtailment to Maximize the Integration of Renewable Energy Generation in Power Systems," Energies, MDPI, vol. 13(16), pages 1-22, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    2. Omar al-Ani & Sanjoy Das & Hongyu Wu, 2023. "Imitation Learning with Deep Attentive Tabular Neural Networks for Environmental Prediction and Control in Smart Home," Energies, MDPI, vol. 16(13), pages 1-19, June.
    3. Guillermo Bejarano & João M. Lemos & Javier Rico-Azagra & Francisco R. Rubio & Manuel G. Ortega, 2022. "Energy Management of Refrigeration Systems with Thermal Energy Storage Based on Non-Linear Model Predictive Control," Mathematics, MDPI, vol. 10(17), pages 1-27, September.
    4. Prasertsak Charoen & Nathavuth Kitbutrawat & Jasada Kudtongngam, 2022. "A Demand Response Implementation with Building Energy Management System," Energies, MDPI, vol. 15(3), pages 1-21, February.
    5. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    6. Dongsu Kim & Jongman Lee & Sunglok Do & Pedro J. Mago & Kwang Ho Lee & Heejin Cho, 2022. "Energy Modeling and Model Predictive Control for HVAC in Buildings: A Review of Current Research Trends," Energies, MDPI, vol. 15(19), pages 1-30, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohammed Awad Abuhussain & Badr Saad Alotaibi & Yakubu Aminu Dodo & Ammar Maghrabi & Muhammad Saidu Aliero, 2024. "Multimodal Framework for Smart Building Occupancy Detection," Sustainability, MDPI, vol. 16(10), pages 1-26, May.
    2. Wenxiao Chu & Maria Vicidomini & Francesco Calise & Neven Duić & Poul Alborg Østergaard & Qiuwang Wang & Maria da Graça Carvalho, 2022. "Recent Advances in Low-Carbon and Sustainable, Efficient Technology: Strategies and Applications," Energies, MDPI, vol. 15(8), pages 1-30, April.
    3. Ahmad, Tanveer & Chen, Huanxin, 2019. "Deep learning for multi-scale smart energy forecasting," Energy, Elsevier, vol. 175(C), pages 98-112.
    4. Jihye Ryu & Jungsoo Kim, 2021. "Effect of Different HVAC Control Strategies on Thermal Comfort and Adaptive Behavior in High-Rise Apartments," Sustainability, MDPI, vol. 13(21), pages 1-20, October.
    5. Stefano Villa & Claudio Sassanelli, 2020. "The Data-Driven Multi-Step Approach for Dynamic Estimation of Buildings’ Interior Temperature," Energies, MDPI, vol. 13(24), pages 1-23, December.
    6. Tarragona, Joan & Pisello, Anna Laura & Fernández, Cèsar & Cabeza, Luisa F. & Payá, Jorge & Marchante-Avellaneda, Javier & de Gracia, Alvaro, 2022. "Analysis of thermal energy storage tanks and PV panels combinations in different buildings controlled through model predictive control," Energy, Elsevier, vol. 239(PC).
    7. Marek Borowski & Klaudia Zwolińska & Marcin Czerwiński, 2022. "An Experimental Study of Thermal Comfort and Indoor Air Quality—A Case Study of a Hotel Building," Energies, MDPI, vol. 15(6), pages 1-18, March.
    8. Iria, José & Soares, Filipe & Matos, Manuel, 2019. "Optimal bidding strategy for an aggregator of prosumers in energy and secondary reserve markets," Applied Energy, Elsevier, vol. 238(C), pages 1361-1372.
    9. Dong, Zihang & Zhang, Xi & Li, Yijun & Strbac, Goran, 2023. "Values of coordinated residential space heating in demand response provision," Applied Energy, Elsevier, vol. 330(PB).
    10. Png, Ethan & Srinivasan, Seshadhri & Bekiroglu, Korkut & Chaoyang, Jiang & Su, Rong & Poolla, Kameshwar, 2019. "An internet of things upgrade for smart and scalable heating, ventilation and air-conditioning control in commercial buildings," Applied Energy, Elsevier, vol. 239(C), pages 408-424.
    11. Nilofar Asim & Marzieh Badiei & Masita Mohammad & Halim Razali & Armin Rajabi & Lim Chin Haw & Mariyam Jameelah Ghazali, 2022. "Sustainability of Heating, Ventilation and Air-Conditioning (HVAC) Systems in Buildings—An Overview," IJERPH, MDPI, vol. 19(2), pages 1-16, January.
    12. Liu, Xuefeng & Xu, Jinman & Bi, Mengbo & Ma, Wenjing & Chen, Wencong & Zheng, Minglong, 2024. "Multivariate coupled full-case physical model of large chilled water systems and its application," Energy, Elsevier, vol. 298(C).
    13. Sheng-Yuan Wang & Kyung-Tae Lee & Ju-Hyung Kim, 2022. "Green Retrofitting Simulation for Sustainable Commercial Buildings in China Using a Proposed Multi-Agent Evolutionary Game," Sustainability, MDPI, vol. 14(13), pages 1-32, June.
    14. Filipe Soares & André Madureira & Andreu Pagès & António Barbosa & António Coelho & Fernando Cassola & Fernando Ribeiro & João Viana & José Andrade & Marina Dorokhova & Nélson Morais & Nicolas Wyrsch , 2021. "FEEdBACk: An ICT-Based Platform to Increase Energy Efficiency through Buildings’ Consumer Engagement," Energies, MDPI, vol. 14(6), pages 1-43, March.
    15. Xiao, Xiangsheng & Wang, JianXiao & Hill, David J., 2022. "Impact of Large-scale concentrated solar power on energy and auxiliary markets," Applied Energy, Elsevier, vol. 318(C).
    16. Cheng, Fanyong & Cui, Can & Cai, Wenjian & Zhang, Xin & Ge, Yuan & Li, Bingxu, 2022. "A novel data-driven air balancing method with energy-saving constraint strategy to minimize the energy consumption of ventilation system," Energy, Elsevier, vol. 239(PB).
    17. Yuri Bulatov & Andrey Kryukov & Andrey Batuhtin & Konstantin Suslov & Ksenia Korotkova & Denis Sidorov, 2022. "Digital Twin Formation Method for Distributed Generation Plants of Cyber–Physical Power Supply Systems," Mathematics, MDPI, vol. 10(16), pages 1-19, August.
    18. Baldi, Simone & Korkas, Christos D. & Lv, Maolong & Kosmatopoulos, Elias B., 2018. "Automating occupant-building interaction via smart zoning of thermostatic loads: A switched self-tuning approach," Applied Energy, Elsevier, vol. 231(C), pages 1246-1258.
    19. Greve, Thomas & Teng, Fei & Pollitt, Michael G. & Strbac, Goran, 2018. "A system operator’s utility function for the frequency response market," Applied Energy, Elsevier, vol. 231(C), pages 562-569.
    20. Darowicki, K. & Janicka, E. & Mielniczek, M. & Zielinski, A. & Gawel, L. & Mitzel, J. & Hunger, J., 2019. "The influence of dynamic load changes on temporary impedance in hydrogen fuel cells, selection and validation of the electrical equivalent circuit," Applied Energy, Elsevier, vol. 251(C), pages 1-1.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:24:p:8210-:d:697081. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.