IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v239y2019icp598-609.html
   My bibliography  Save this article

Optimal energy management strategies for energy Internet via deep reinforcement learning approach

Author

Listed:
  • Hua, Haochen
  • Qin, Yuchao
  • Hao, Chuantong
  • Cao, Junwei

Abstract

This paper investigates the energy management problem in the field of energy Internet (EI) with interdisciplinary techniques. The concept of EI has been proposed for a while. However, there still exist many fundamental and technical issues that have not been fully investigated. In this paper, a new energy regulation issue is considered based on the operational principles of EI. Multiple targets are considered along with constraints. Then, the practical energy management problem is formulated as a constrained optimal control problem. Notably, no explicit mathematical model for power of renewable power generation devices and loads is utilized. Due to the complexity of this problem, conventional methods appear to be inapplicable. To obtain the desired control scheme, a model-free deep reinforcement learning algorithm is applied. A practical solution is obtained, and the feasibility as well as the performance of the proposed method are evaluated with numerical simulations.

Suggested Citation

  • Hua, Haochen & Qin, Yuchao & Hao, Chuantong & Cao, Junwei, 2019. "Optimal energy management strategies for energy Internet via deep reinforcement learning approach," Applied Energy, Elsevier, vol. 239(C), pages 598-609.
    • Handle: RePEc:eee:appene:v:239:y:2019:i:c:p:598-609
    DOI: 10.1016/j.apenergy.2019.01.145
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261919301746
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2019.01.145?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mathiesen, B.V. & Lund, H. & Connolly, D. & Wenzel, H. & Østergaard, P.A. & Möller, B. & Nielsen, S. & Ridjan, I. & Karnøe, P. & Sperling, K. & Hvelplund, F.K., 2015. "Smart Energy Systems for coherent 100% renewable energy and transport solutions," Applied Energy, Elsevier, vol. 145(C), pages 139-154.
    2. Kou, Peng & Liang, Deliang & Gao, Lin, 2017. "Distributed EMPC of multiple microgrids for coordinated stochastic energy management," Applied Energy, Elsevier, vol. 185(P1), pages 939-952.
    3. Vachirasricirikul, Sitthidet & Ngamroo, Issarachai, 2011. "Robust controller design of heat pump and plug-in hybrid electric vehicle for frequency control in a smart microgrid based on specified-structure mixed H2/H∞ control technique," Applied Energy, Elsevier, vol. 88(11), pages 3860-3868.
    4. Honglu Zhu & Xu Li & Qiao Sun & Ling Nie & Jianxi Yao & Gang Zhao, 2015. "A Power Prediction Method for Photovoltaic Power Plant Based on Wavelet Decomposition and Artificial Neural Networks," Energies, MDPI, vol. 9(1), pages 1-15, December.
    5. Korkas, Christos D. & Baldi, Simone & Michailidis, Iakovos & Kosmatopoulos, Elias B., 2016. "Occupancy-based demand response and thermal comfort optimization in microgrids with renewable energy sources and energy storage," Applied Energy, Elsevier, vol. 163(C), pages 93-104.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. José Luis Ruiz Duarte & Neng Fan, 2022. "Operation of a Power Grid with Embedded Networked Microgrids and Onsite Renewable Technologies," Energies, MDPI, vol. 15(7), pages 1-24, March.
    2. Yamashita, Daniela Yassuda & Vechiu, Ionel & Gaubert, Jean-Paul, 2020. "A review of hierarchical control for building microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 118(C).
    3. Haochen Hua & Chuantong Hao & Yuchao Qin & Junwei Cao, 2018. "A Class of Control Strategies for Energy Internet Considering System Robustness and Operation Cost Optimization," Energies, MDPI, vol. 11(6), pages 1-20, June.
    4. Kostevšek, Anja & Klemeš, Jiří Jaromír & Varbanov, Petar Sabev & Papa, Gregor & Petek, Janez, 2016. "The concept of an ecosystem model to support the transformation to sustainable energy systems," Applied Energy, Elsevier, vol. 184(C), pages 1460-1469.
    5. Bogdanov, Dmitrii & Breyer, Christian, 2024. "Role of smart charging of electric vehicles and vehicle-to-grid in integrated renewables-based energy systems on country level," Energy, Elsevier, vol. 301(C).
    6. Østergaard, P.A. & Lund, H. & Thellufsen, J.Z. & Sorknæs, P. & Mathiesen, B.V., 2022. "Review and validation of EnergyPLAN," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    7. Lund, Henrik & Thellufsen, Jakob Zinck & Sorknæs, Peter & Mathiesen, Brian Vad & Chang, Miguel & Madsen, Poul Thøis & Kany, Mikkel Strunge & Skov, Iva Ridjan, 2022. "Smart energy Denmark. A consistent and detailed strategy for a fully decarbonized society," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    8. Persson, Urban & Wiechers, Eva & Möller, Bernd & Werner, Sven, 2019. "Heat Roadmap Europe: Heat distribution costs," Energy, Elsevier, vol. 176(C), pages 604-622.
    9. Østergaard, Poul Alberg & Werner, Sven & Dyrelund, Anders & Lund, Henrik & Arabkoohsar, Ahmad & Sorknæs, Peter & Gudmundsson, Oddgeir & Thorsen, Jan Eric & Mathiesen, Brian Vad, 2022. "The four generations of district cooling - A categorization of the development in district cooling from origin to future prospect," Energy, Elsevier, vol. 253(C).
    10. Guelpa, Elisa & Bischi, Aldo & Verda, Vittorio & Chertkov, Michael & Lund, Henrik, 2019. "Towards future infrastructures for sustainable multi-energy systems: A review," Energy, Elsevier, vol. 184(C), pages 2-21.
    11. Kovacic, Zora & Giampietro, Mario, 2015. "Empty promises or promising futures? The case of smart grids," Energy, Elsevier, vol. 93(P1), pages 67-74.
    12. Eissa, M.M., 2018. "First time real time incentive demand response program in smart grid with “i-Energy” management system with different resources," Applied Energy, Elsevier, vol. 212(C), pages 607-621.
    13. Khosravi, Fatemeh & Lowes, Richard & Ugalde-Loo, Carlos E., 2023. "Cooling is hotting up in the UK," Energy Policy, Elsevier, vol. 174(C).
    14. Wang, Yongli & Li, Jiapu & Wang, Shuo & Yang, Jiale & Qi, Chengyuan & Guo, Hongzhen & Liu, Ximei & Zhang, Hongqing, 2020. "Operational optimization of wastewater reuse integrated energy system," Energy, Elsevier, vol. 200(C).
    15. Maruf, Md. Nasimul Islam, 2021. "Open model-based analysis of a 100% renewable and sector-coupled energy system–The case of Germany in 2050," Applied Energy, Elsevier, vol. 288(C).
    16. Besagni, Giorgio & Premoli Vilà, Lidia & Borgarello, Marco & Trabucchi, Andrea & Merlo, Marco & Rodeschini, Jacopo & Finazzi, Francesco, 2021. "Electrification pathways of the Italian residential sector under socio-demographic constrains: Looking towards 2040," Energy, Elsevier, vol. 217(C).
    17. Grundahl, Lars & Nielsen, Steffen & Lund, Henrik & Möller, Bernd, 2016. "Comparison of district heating expansion potential based on consumer-economy or socio-economy," Energy, Elsevier, vol. 115(P3), pages 1771-1778.
    18. Aleksandra Matuszewska-Janica & Dorota Żebrowska-Suchodolska & Urszula Ala-Karvia & Marta Hozer-Koćmiel, 2021. "Changes in Electricity Production from Renewable Energy Sources in the European Union Countries in 2005–2019," Energies, MDPI, vol. 14(19), pages 1-27, October.
    19. Iribarren, Diego & Martín-Gamboa, Mario & Navas-Anguita, Zaira & García-Gusano, Diego & Dufour, Javier, 2020. "Influence of climate change externalities on the sustainability-oriented prioritisation of prospective energy scenarios," Energy, Elsevier, vol. 196(C).
    20. Song, Kwonsik & Kim, Sooyoung & Park, Moonseo & Lee, Hyun-Soo, 2017. "Energy efficiency-based course timetabling for university buildings," Energy, Elsevier, vol. 139(C), pages 394-405.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:239:y:2019:i:c:p:598-609. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.