IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i23p9032-d988076.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Operation of Transmission Battery Storage with Dynamic Thermal Line Rating

Author

Listed:
  • Vadim Avkhimenia

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Matheus Gemignani

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Tim Weis

    (Mechanical Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada)

  • Petr Musilek

    (Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada
    Applied Cybernetics, University of Hradec Králové, 500 03 Hradec Králové, Czech Republic)

Abstract

It is well known that dynamic thermal line rating has the potential to use power transmission infrastructure more effectively by allowing higher currents when lines are cooler; however, it is not commonly implemented. Some of the barriers to implementation can be mitigated using modern battery energy storage systems. This paper proposes a combination of dynamic thermal line rating and battery use through the application of deep reinforcement learning. In particular, several algorithms based on deep deterministic policy gradient and soft actor critic are examined, in both single- and multi-agent settings. The selected algorithms are used to control battery energy storage systems in a 6-bus test grid. The effects of load and transmissible power forecasting on the convergence of those algorithms are also examined. The soft actor critic algorithm performs best, followed by deep deterministic policy gradient, and their multi-agent versions in the same order. One-step forecasting of the load and ampacity does not provide any significant benefit for predicting battery action.

Suggested Citation

  • Vadim Avkhimenia & Matheus Gemignani & Tim Weis & Petr Musilek, 2022. "Deep Reinforcement Learning-Based Operation of Transmission Battery Storage with Dynamic Thermal Line Rating," Energies, MDPI, vol. 15(23), pages 1-15, November.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:23:p:9032-:d:988076
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/23/9032/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/23/9032/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Holger C. Hesse & Rodrigo Martins & Petr Musilek & Maik Naumann & Cong Nam Truong & Andreas Jossen, 2017. "Economic Optimization of Component Sizing for Residential Battery Storage Systems," Energies, MDPI, vol. 10(7), pages 1-19, June.
    2. Akhtar Hussain & Van-Hai Bui & Hak-Man Kim, 2017. "Impact Analysis of Demand Response Intensity and Energy Storage Size on Operation of Networked Microgrids," Energies, MDPI, vol. 10(7), pages 1-19, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ghorbanzadeh, Milad & Astaneh, Majid & Golzar, Farzin, 2019. "Long-term degradation based analysis for lithium-ion batteries in off-grid wind-battery renewable energy systems," Energy, Elsevier, vol. 166(C), pages 1194-1206.
    2. Pusceddu, Elian & Zakeri, Behnam & Castagneto Gissey, Giorgio, 2021. "Synergies between energy arbitrage and fast frequency response for battery energy storage systems," Applied Energy, Elsevier, vol. 283(C).
    3. Andrea Mazza & Hamidreza Mirtaheri & Gianfranco Chicco & Angela Russo & Maurizio Fantino, 2019. "Location and Sizing of Battery Energy Storage Units in Low Voltage Distribution Networks," Energies, MDPI, vol. 13(1), pages 1-20, December.
    4. Beuse, Martin & Dirksmeier, Mathias & Steffen, Bjarne & Schmidt, Tobias S., 2020. "Profitability of commercial and industrial photovoltaics and battery projects in South-East-Asia," Applied Energy, Elsevier, vol. 271(C).
    5. Liu, Jia & Chen, Xi & Yang, Hongxing & Li, Yutong, 2020. "Energy storage and management system design optimization for a photovoltaic integrated low-energy building," Energy, Elsevier, vol. 190(C).
    6. Mulleriyawage, U.G.K. & Shen, W.X., 2021. "Impact of demand side management on optimal sizing of residential battery energy storage system," Renewable Energy, Elsevier, vol. 172(C), pages 1250-1266.
    7. Jiang, Yinghua & Kang, Lixia & Liu, Yongzhong, 2019. "A unified model to optimize configuration of battery energy storage systems with multiple types of batteries," Energy, Elsevier, vol. 176(C), pages 552-560.
    8. Wu, Wei & Lin, Boqiang, 2018. "Application value of energy storage in power grid: A special case of China electricity market," Energy, Elsevier, vol. 165(PB), pages 1191-1199.
    9. Warren S. Vaz, 2020. "Multiobjective Optimization of a Residential Grid-Tied Solar System," Sustainability, MDPI, vol. 12(20), pages 1-15, October.
    10. Hector Beltran & Pablo Ayuso & Emilio Pérez, 2020. "Lifetime Expectancy of Li-Ion Batteries used for Residential Solar Storage," Energies, MDPI, vol. 13(3), pages 1-18, January.
    11. Juha Koskela & Antti Mutanen & Pertti Järventausta, 2020. "Using Load Forecasting to Control Domestic Battery Energy Storage Systems," Energies, MDPI, vol. 13(15), pages 1-20, August.
    12. Laugs, Gideon A.H. & Benders, René M.J. & Moll, Henri C., 2024. "Maximizing self-sufficiency and minimizing grid interaction: Combining electric and molecular energy storage for decentralized balancing of variable renewable energy in local energy systems," Renewable Energy, Elsevier, vol. 229(C).
    13. Chatzisideris, Marios D. & Ohms, Pernille K. & Espinosa, Nieves & Krebs, Frederik C. & Laurent, Alexis, 2019. "Economic and environmental performances of organic photovoltaics with battery storage for residential self-consumption," Applied Energy, Elsevier, vol. 256(C).
    14. Tervo, Eric & Agbim, Kenechi & DeAngelis, Freddy & Hernandez, Jeffrey & Kim, Hye Kyung & Odukomaiya, Adewale, 2018. "An economic analysis of residential photovoltaic systems with lithium ion battery storage in the United States," Renewable and Sustainable Energy Reviews, Elsevier, vol. 94(C), pages 1057-1066.
    15. Despeghel, Jolien & Tant, Jeroen & Driesen, Johan, 2024. "Convex optimization of PV-battery system sizing and operation with non-linear loss models," Applied Energy, Elsevier, vol. 353(PA).
    16. Alejandro Pena-Bello & Edward Barbour & Marta C. Gonzalez & Selin Yilmaz & Martin K. Patel & David Parra, 2020. "How Does the Electricity Demand Profile Impact the Attractiveness of PV-Coupled Battery Systems Combining Applications?," Energies, MDPI, vol. 13(15), pages 1-19, August.
    17. Wu, Yaling & Liu, Zhongbing & Liu, Jiangyang & Xiao, Hui & Liu, Ruimiao & Zhang, Ling, 2022. "Optimal battery capacity of grid-connected PV-battery systems considering battery degradation," Renewable Energy, Elsevier, vol. 181(C), pages 10-23.
    18. Lucas Deotti & Wanessa Guedes & Bruno Dias & Tiago Soares, 2020. "Technical and Economic Analysis of Battery Storage for Residential Solar Photovoltaic Systems in the Brazilian Regulatory Context," Energies, MDPI, vol. 13(24), pages 1-30, December.
    19. Frida Berglund & Salman Zaferanlouei & Magnus Korpås & Kjetil Uhlen, 2019. "Optimal Operation of Battery Storage for a Subscribed Capacity-Based Power Tariff Prosumer—A Norwegian Case Study," Energies, MDPI, vol. 12(23), pages 1-24, November.
    20. Mihai Sanduleac & Irina Ciornei & Mihaela Albu & Lucian Toma & Marta Sturzeanu & João F. Martins, 2017. "Resilient Prosumer Scenario in a Changing Regulatory Environment—The UniRCon Solution," Energies, MDPI, vol. 10(12), pages 1-22, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:23:p:9032-:d:988076. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.