IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v377y2025ipbs0306261924019573.html
   My bibliography  Save this article

Reinforcement learning for heliostat aiming: Improving the performance of Solar Tower plants

Author

Listed:
  • Carballo, J.A.
  • Bonilla, J.
  • Cruz, N.C.
  • Fernández-Reche, J.
  • Álvarez, J.D.
  • Avila-Marin, A.
  • Berenguel, M.

Abstract

Solar Tower (ST) systems use heliostats to concentrate solar radiation onto a tower-mounted receiver. Optimizing the aiming strategy for these heliostats over the receiver remains a critical challenge due to the dynamic nature of solar radiation and the need to maximize energy capture while ensuring operational safety. This paper introduces a novel, model-free deep Reinforcement Learning (RL) approach to optimize heliostat aiming strategies, utilizing the Soft Actor–Critic (SAC) algorithm. This advanced RL method enhances the traditional Actor–Critic framework with two neural networks. The proposal dynamically adjusts the aiming points across the receiver surface in real time, trying to improve the overall performance of the ST plant. The strategy was simulated and evaluated over a full operational year and compared with traditional methods. The results show an increase of more than 8.8% in yearly absorbed power, a significant improvement that directly enhances performance and contributes to better economic outcomes for the technology. This technique also eliminates the need for constant human intervention and is applicable to both existing and future plants.

Suggested Citation

  • Carballo, J.A. & Bonilla, J. & Cruz, N.C. & Fernández-Reche, J. & Álvarez, J.D. & Avila-Marin, A. & Berenguel, M., 2025. "Reinforcement learning for heliostat aiming: Improving the performance of Solar Tower plants," Applied Energy, Elsevier, vol. 377(PB).
    • Handle: RePEc:eee:appene:v:377:y:2025:i:pb:s0306261924019573
    DOI: 10.1016/j.apenergy.2024.124574
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924019573
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.124574?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Siva Reddy, V. & Kaushik, S.C. & Ranjan, K.R. & Tyagi, S.K., 2013. "State-of-the-art of solar thermal power plants—A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 258-273.
    2. Spiros Alexopoulos & Bernhard Hoffschmidt, 2017. "Advances in solar tower technology," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 6(1), January.
    3. Zeng, Zhichen & Ni, Dong & Xiao, Gang, 2022. "Real-time heliostat field aiming strategy optimization based on reinforcement learning," Applied Energy, Elsevier, vol. 307(C).
    4. Cruz, N.C. & Salhi, S. & Redondo, J.L. & Álvarez, J.D. & Berenguel, M. & Ortigosa, P.M., 2018. "Hector, a new methodology for continuous and pattern-free heliostat field optimization," Applied Energy, Elsevier, vol. 225(C), pages 1123-1131.
    5. Wang, Jianxing & Duan, Liqiang & Yang, Yongping, 2018. "An improvement crossover operation method in genetic algorithm and spatial optimization of heliostat field," Energy, Elsevier, vol. 155(C), pages 15-28.
    6. Saghafifar, Mohammad & Gadalla, Mohamed & Mohammadi, Kasra, 2019. "Thermo-economic analysis and optimization of heliostat fields using AINEH code: Analysis of implementation of non-equal heliostats (AINEH)," Renewable Energy, Elsevier, vol. 135(C), pages 920-935.
    7. Ruidi Zhu & Dong Ni, 2023. "A Model Predictive Control Approach for Heliostat Field Power Regulatory Aiming Strategy under Varying Cloud Shadowing Conditions," Energies, MDPI, vol. 16(7), pages 1-19, March.
    8. Benjamin Grange & Gilles Flamant, 2021. "Aiming Strategy on a Prototype-Scale Solar Receiver: Coupling of Tabu Search, Ray-Tracing and Thermal Models," Sustainability, MDPI, vol. 13(7), pages 1-22, April.
    9. Yu, Qiang & Wang, Zhifeng & Xu, Ershu, 2014. "Analysis and improvement of solar flux distribution inside a cavity receiver based on multi-focal points of heliostat field," Applied Energy, Elsevier, vol. 136(C), pages 417-430.
    10. Laporte-Azcué, M. & Rodríguez-Sánchez, M.R. & González-Gómez, P.A. & Santana, D., 2021. "Assessment of the time resolution used to estimate the central solar receiver lifetime," Applied Energy, Elsevier, vol. 301(C).
    11. Kuhnke, Sascha & Richter, Pascal & Kepp, Fynn & Cumpston, Jeff & Koster, Arie M.C.A. & Büsing, Christina, 2020. "Robust optimal aiming strategies in central receiver systems," Renewable Energy, Elsevier, vol. 152(C), pages 198-207.
    12. Speetzen, N. & Richter, P., 2021. "Dynamic aiming strategy for central receiver systems," Renewable Energy, Elsevier, vol. 180(C), pages 55-67.
    13. Cruz, N.C. & Redondo, J.L. & Berenguel, M. & Álvarez, J.D. & Ortigosa, P.M., 2017. "Review of software for optical analyzing and optimizing heliostat fields," Renewable and Sustainable Energy Reviews, Elsevier, vol. 72(C), pages 1001-1018.
    14. Ashley, Thomas & Carrizosa, Emilio & Fernández-Cara, Enrique, 2017. "Optimisation of aiming strategies in Solar Power Tower plants," Energy, Elsevier, vol. 137(C), pages 285-291.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zeng, Zhichen & Ni, Dong & Xiao, Gang, 2022. "Real-time heliostat field aiming strategy optimization based on reinforcement learning," Applied Energy, Elsevier, vol. 307(C).
    2. Ruidi Zhu & Dong Ni, 2023. "A Model Predictive Control Approach for Heliostat Field Power Regulatory Aiming Strategy under Varying Cloud Shadowing Conditions," Energies, MDPI, vol. 16(7), pages 1-19, March.
    3. Arrif, Toufik & Hassani, Samir & Guermoui, Mawloud & Sánchez-González, A. & A.Taylor, Robert & Belaid, Abdelfetah, 2022. "GA-GOA hybrid algorithm and comparative study of different metaheuristic population-based algorithms for solar tower heliostat field design," Renewable Energy, Elsevier, vol. 192(C), pages 745-758.
    4. Sánchez-González, Alberto & Kontopyrgos, Marios & Milidonis, Kypros & Georgiou, Marios C., 2024. "Heliostat field aiming strategy based on deterministic optimization: An experimental validation," Renewable Energy, Elsevier, vol. 236(C).
    5. Sánchez-González, Alberto & Rodríguez-Sánchez, María Reyes & Santana, Domingo, 2018. "Aiming factor to flatten the flux distribution on cylindrical receivers," Energy, Elsevier, vol. 153(C), pages 113-125.
    6. Georgios E. Arnaoutakis & Dimitris Al. Katsaprakakis, 2021. "Concentrating Solar Power Advances in Geometric Optics, Materials and System Integration," Energies, MDPI, vol. 14(19), pages 1-25, September.
    7. García, Jesús & Barraza, Rodrigo & Soo Too, Yen Chean & Vásquez-Padilla, Ricardo & Acosta, David & Estay, Danilo & Valdivia, Patricio, 2022. "Transient simulation of a control strategy for solar receivers based on mass flow valves adjustments and heliostats aiming," Renewable Energy, Elsevier, vol. 185(C), pages 1221-1244.
    8. Ashikuzzaman, A.K.M. & Adnan, Sakib, 2021. "Optical efficiency comparison of circular heliostat fields: Engender of hybrid layouts," Renewable Energy, Elsevier, vol. 178(C), pages 506-519.
    9. Merchán, R.P. & Santos, M.J. & Medina, A. & Calvo Hernández, A., 2022. "High temperature central tower plants for concentrated solar power: 2021 overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 155(C).
    10. Collado, Francisco J. & Guallar, Jesus, 2019. "Quick design of regular heliostat fields for commercial solar tower power plants," Energy, Elsevier, vol. 178(C), pages 115-125.
    11. Yerudkar, Aditi N. & Kumar, Durgesh & Dalvi, Vishwanath H. & Panse, Sudhir V. & Gaval, Vivek R. & Joshi, Jyeshtharaj B., 2024. "Economically feasible solutions in concentrating solar power technology specifically for heliostats – A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    12. Cui, Dongyu & Bian, Hong & Yu, Haizheng, 2024. "Quickly select heliostat candidates and design pattern-free layout using geometric projection method," Renewable Energy, Elsevier, vol. 237(PB).
    13. Cruz, N.C. & Salhi, S. & Redondo, J.L. & Álvarez, J.D. & Berenguel, M. & Ortigosa, P.M., 2018. "Hector, a new methodology for continuous and pattern-free heliostat field optimization," Applied Energy, Elsevier, vol. 225(C), pages 1123-1131.
    14. Ghirardi, Elisa & Brumana, Giovanni & Franchini, Giuseppe & Perdichizzi, Antonio, 2021. "Heliostat layout optimization for load-following solar tower plants," Renewable Energy, Elsevier, vol. 168(C), pages 393-405.
    15. Ashley, Thomas & Carrizosa, Emilio & Fernández-Cara, Enrique, 2019. "Heliostat field cleaning scheduling for Solar Power Tower plants: A heuristic approach," Applied Energy, Elsevier, vol. 235(C), pages 653-660.
    16. Okoroigwe, Edmund & Madhlopa, Amos, 2016. "An integrated combined cycle system driven by a solar tower: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 337-350.
    17. Merad, Faycel & Labar, Hocine & Samira KELAIAIA, Mounia & Necaibia, Salah & Djelailia, Okba, 2019. "A maximum power control based on flexible collector applied to concentrator solar power," Renewable and Sustainable Energy Reviews, Elsevier, vol. 110(C), pages 315-331.
    18. García, Jesús & Barraza, Rodrigo & Soo Too, Yen Chean & Vásquez Padilla, Ricardo & Acosta, David & Estay, Danilo & Valdivia, Patricio, 2020. "Aiming clusters of heliostats over solar receivers for distributing heat flux using one variable per group," Renewable Energy, Elsevier, vol. 160(C), pages 584-596.
    19. Wang, Jianxing & Duan, Liqiang & Yang, Yongping, 2018. "An improvement crossover operation method in genetic algorithm and spatial optimization of heliostat field," Energy, Elsevier, vol. 155(C), pages 15-28.
    20. Cioccolanti, Luca & Tascioni, Roberto & Arteconi, Alessia, 2018. "Mathematical modelling of operation modes and performance evaluation of an innovative small-scale concentrated solar organic Rankine cycle plant," Applied Energy, Elsevier, vol. 221(C), pages 464-476.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:377:y:2025:i:pb:s0306261924019573. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.