IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i7p2607-d786086.html
   My bibliography  Save this article

Control of a PVT-Heat-Pump-System Based on Reinforcement Learning–Operating Cost Reduction through Flow Rate Variation

Author

Listed:
  • Daniel John

    (Institute for Environmental Technology and Energy Economics, Hamburg University of Technology, Eissendorfer Strasse 40, 21073 Hamburg, Germany)

  • Martin Kaltschmitt

    (Institute for Environmental Technology and Energy Economics, Hamburg University of Technology, Eissendorfer Strasse 40, 21073 Hamburg, Germany)

Abstract

This study aims to develop a controller to operate an energy system-consisting of a photovoltaic thermal (PVT) system combined with a heat pump, using the reinforcement learning approach to minimize the operating costs of the system. For this, the flow rate of the cooling fluid pumped through the PVT system is controlled. This flow rate determines the temperature increase of the cooling fluid while reducing the temperature of the PVT system. The heated-up cooling fluid is used to improve the heat pump’s coefficient of performance (COP). For optimizing the operation costs of such a system, first an extensive simulation model has been developed. Based on this technical model, a controller has been developed using the reinforcement learning approach to allow for a cost-efficient control of the flow rate. The results show that a successfully trained control unit based on the reinforcement learning approach can reduce the operating costs with an independent validation dataset. For the case study presented here, based on the implemented methodological approach, including hyperparameter optimization, the operating costs of the investigated energy system can be reduced by more than 4% in the training dataset and by close to 3% in the validation dataset.

Suggested Citation

  • Daniel John & Martin Kaltschmitt, 2022. "Control of a PVT-Heat-Pump-System Based on Reinforcement Learning–Operating Cost Reduction through Flow Rate Variation," Energies, MDPI, vol. 15(7), pages 1-19, April.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:7:p:2607-:d:786086
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/7/2607/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/7/2607/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yang, Lei & Nagy, Zoltan & Goffin, Philippe & Schlueter, Arno, 2015. "Reinforcement learning for optimal control of low exergy buildings," Applied Energy, Elsevier, vol. 156(C), pages 577-586.
    2. Madalina Barbu & George Darie & Monica Siroux, 2020. "A Parametric Study of a Hybrid Photovoltaic Thermal (PVT) System Coupled with a Domestic Hot Water (DHW) Storage Tank," Energies, MDPI, vol. 13(24), pages 1-18, December.
    3. Sree Harsha Bandaru & Victor Becerra & Sourav Khanna & Jovana Radulovic & David Hutchinson & Rinat Khusainov, 2021. "A Review of Photovoltaic Thermal (PVT) Technology for Residential Applications: Performance Indicators, Progress, and Opportunities," Energies, MDPI, vol. 14(13), pages 1-48, June.
    4. Joshi, Sandeep S. & Dhoble, Ashwinkumar S., 2018. "Photovoltaic -Thermal systems (PVT): Technology review and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 92(C), pages 848-882.
    5. Pfenninger, Stefan & Staffell, Iain, 2016. "Long-term patterns of European PV output using 30 years of validated hourly reanalysis and satellite data," Energy, Elsevier, vol. 114(C), pages 1251-1265.
    6. Sathe, Tushar M. & Dhoble, A.S., 2017. "A review on recent advancements in photovoltaic thermal techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 76(C), pages 645-672.
    7. Lamnatou, Chr. & Chemisana, D., 2017. "Photovoltaic/thermal (PVT) systems: A review with emphasis on environmental issues," Renewable Energy, Elsevier, vol. 105(C), pages 270-287.
    8. Yu, Ying & Long, Enshen & Chen, Xi & Yang, Hongxing, 2019. "Testing and modelling an unglazed photovoltaic thermal collector for application in Sichuan Basin," Applied Energy, Elsevier, vol. 242(C), pages 931-941.
    9. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    10. Hossain, M.S. & Pandey, A.K. & Selvaraj, Jeyraj & Rahim, Nasrudin Abd & Islam, M.M. & Tyagi, V.V., 2019. "Two side serpentine flow based photovoltaic-thermal-phase change materials (PVT-PCM) system: Energy, exergy and economic analysis," Renewable Energy, Elsevier, vol. 136(C), pages 1320-1336.
    11. Staffell, Iain & Pfenninger, Stefan, 2016. "Using bias-corrected reanalysis to simulate current and future wind power output," Energy, Elsevier, vol. 114(C), pages 1224-1239.
    12. Herrando, María & Pantaleo, Antonio M. & Wang, Kai & Markides, Christos N., 2019. "Solar combined cooling, heating and power systems based on hybrid PVT, PV or solar-thermal collectors for building applications," Renewable Energy, Elsevier, vol. 143(C), pages 637-647.
    13. Vallati, A. & Ocłoń, P. & Colucci, C. & Mauri, L. & de Lieto Vollaro, R. & Taler, J., 2019. "Energy analysis of a thermal system composed by a heat pump coupled with a PVT solar collector," Energy, Elsevier, vol. 174(C), pages 91-96.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kamel Guedri & Mohamed Salem & Mamdouh El Haj Assad & Jaroon Rungamornrat & Fatimah Malek Mohsen & Yonis M. Buswig, 2022. "PV/Thermal as Promising Technologies in Buildings: A Comprehensive Review on Exergy Analysis," Sustainability, MDPI, vol. 14(19), pages 1-16, September.
    2. Pang, Wei & Cui, Yanan & Zhang, Qian & Wilson, Gregory.J. & Yan, Hui, 2020. "A comparative analysis on performances of flat plate photovoltaic/thermal collectors in view of operating media, structural designs, and climate conditions," Renewable and Sustainable Energy Reviews, Elsevier, vol. 119(C).
    3. Mukhamad Faeshol Umam & Md. Hasanuzzaman & Nasrudin Abd Rahim, 2022. "Global Advancement of Nanofluid-Based Sheet and Tube Collectors for a Photovoltaic Thermal System," Energies, MDPI, vol. 15(15), pages 1-37, August.
    4. Reji Kumar, R. & Samykano, M. & Pandey, A.K. & Kadirgama, K. & Tyagi, V.V., 2020. "Phase change materials and nano-enhanced phase change materials for thermal energy storage in photovoltaic thermal systems: A futuristic approach and its technical challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 133(C).
    5. María Herrando & Alba Ramos, 2022. "Photovoltaic-Thermal (PV-T) Systems for Combined Cooling, Heating and Power in Buildings: A Review," Energies, MDPI, vol. 15(9), pages 1-28, April.
    6. Josué F. Rosales-Pérez & Andrés Villarruel-Jaramillo & José A. Romero-Ramos & Manuel Pérez-García & José M. Cardemil & Rodrigo Escobar, 2023. "Hybrid System of Photovoltaic and Solar Thermal Technologies for Industrial Process Heat," Energies, MDPI, vol. 16(5), pages 1-45, February.
    7. Bugaj, Marcin A. & Mik, Krzysztof, 2023. "Can PVT bend?: The elaboration of flexible hybrid photovoltaic thermal solar collector structure and testing methodology," Renewable Energy, Elsevier, vol. 215(C).
    8. Cirrincione, Laura & Malara, Cristina & Marino, Concettina & Nucara, Antonino & Peri, Giorgia & Pietrafesa, Matilde, 2020. "Effect of the thermal storage dimensions on the performances of solar photovoltaic-thermal systems," Renewable Energy, Elsevier, vol. 162(C), pages 2004-2018.
    9. Sree Harsha Bandaru & Victor Becerra & Sourav Khanna & Jovana Radulovic & David Hutchinson & Rinat Khusainov, 2021. "A Review of Photovoltaic Thermal (PVT) Technology for Residential Applications: Performance Indicators, Progress, and Opportunities," Energies, MDPI, vol. 14(13), pages 1-48, June.
    10. de Guibert, Paul & Shirizadeh, Behrang & Quirion, Philippe, 2020. "Variable time-step: A method for improving computational tractability for energy system models with long-term storage," Energy, Elsevier, vol. 213(C).
    11. Marko Hočevar & Lovrenc Novak & Primož Drešar & Gašper Rak, 2022. "The Status Quo and Future of Hydropower in Slovenia," Energies, MDPI, vol. 15(19), pages 1-13, September.
    12. Lukas Kriechbaum & Philipp Gradl & Romeo Reichenhauser & Thomas Kienberger, 2020. "Modelling Grid Constraints in a Multi-Energy Municipal Energy System Using Cumulative Exergy Consumption Minimisation," Energies, MDPI, vol. 13(15), pages 1-23, July.
    13. Behrang Shirizadeh, Quentin Perrier, and Philippe Quirion, 2022. "How Sensitive are Optimal Fully Renewable Power Systems to Technology Cost Uncertainty?," The Energy Journal, International Association for Energy Economics, vol. 0(Number 1).
    14. Omoyele, Olalekan & Hoffmann, Maximilian & Koivisto, Matti & Larrañeta, Miguel & Weinand, Jann Michael & Linßen, Jochen & Stolten, Detlef, 2024. "Increasing the resolution of solar and wind time series for energy system modeling: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    15. Yao, Jian & Dou, Pengbo & Zheng, Sihang & Zhao, Yao & Dai, Yanjun & Zhu, Junjie & Novakovic, Vojislav, 2022. "Co-generation ability investigation of the novel structured PVT heat pump system and its effect on the “Carbon neutral” strategy of Shanghai," Energy, Elsevier, vol. 239(PA).
    16. Liu, Hailiang & Andresen, Gorm Bruun & Greiner, Martin, 2018. "Cost-optimal design of a simplified highly renewable Chinese electricity network," Energy, Elsevier, vol. 147(C), pages 534-546.
    17. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    18. Khani, M.S. & Baneshi, M. & Eslami, M., 2019. "Bi-objective optimization of photovoltaic-thermal (PV/T) solar collectors according to various weather conditions using genetic algorithm: A numerical modeling," Energy, Elsevier, vol. 189(C).
    19. Yu, Qinghua & Chen, Xi & Yang, Hongxing, 2021. "Research progress on utilization of phase change materials in photovoltaic/thermal systems: A critical review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    20. Géremi Gilson Dranka & Paula Ferreira, 2020. "Electric Vehicles and Biofuels Synergies in the Brazilian Energy System," Energies, MDPI, vol. 13(17), pages 1-22, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:7:p:2607-:d:786086. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.