IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v239y2019icp598-609.html
   My bibliography  Save this article

Optimal energy management strategies for energy Internet via deep reinforcement learning approach

Author

Listed:
  • Hua, Haochen
  • Qin, Yuchao
  • Hao, Chuantong
  • Cao, Junwei

Abstract

This paper investigates the energy management problem in the field of energy Internet (EI) with interdisciplinary techniques. The concept of EI has been proposed for a while. However, there still exist many fundamental and technical issues that have not been fully investigated. In this paper, a new energy regulation issue is considered based on the operational principles of EI. Multiple targets are considered along with constraints. Then, the practical energy management problem is formulated as a constrained optimal control problem. Notably, no explicit mathematical model for power of renewable power generation devices and loads is utilized. Due to the complexity of this problem, conventional methods appear to be inapplicable. To obtain the desired control scheme, a model-free deep reinforcement learning algorithm is applied. A practical solution is obtained, and the feasibility as well as the performance of the proposed method are evaluated with numerical simulations.

Suggested Citation

  • Hua, Haochen & Qin, Yuchao & Hao, Chuantong & Cao, Junwei, 2019. "Optimal energy management strategies for energy Internet via deep reinforcement learning approach," Applied Energy, Elsevier, vol. 239(C), pages 598-609.
    • Handle: RePEc:eee:appene:v:239:y:2019:i:c:p:598-609
    DOI: 10.1016/j.apenergy.2019.01.145
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261919301746
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2019.01.145?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vachirasricirikul, Sitthidet & Ngamroo, Issarachai, 2011. "Robust controller design of heat pump and plug-in hybrid electric vehicle for frequency control in a smart microgrid based on specified-structure mixed H2/H∞ control technique," Applied Energy, Elsevier, vol. 88(11), pages 3860-3868.
    2. Kou, Peng & Liang, Deliang & Gao, Lin, 2017. "Distributed EMPC of multiple microgrids for coordinated stochastic energy management," Applied Energy, Elsevier, vol. 185(P1), pages 939-952.
    3. Honglu Zhu & Xu Li & Qiao Sun & Ling Nie & Jianxi Yao & Gang Zhao, 2015. "A Power Prediction Method for Photovoltaic Power Plant Based on Wavelet Decomposition and Artificial Neural Networks," Energies, MDPI, vol. 9(1), pages 1-15, December.
    4. Korkas, Christos D. & Baldi, Simone & Michailidis, Iakovos & Kosmatopoulos, Elias B., 2016. "Occupancy-based demand response and thermal comfort optimization in microgrids with renewable energy sources and energy storage," Applied Energy, Elsevier, vol. 163(C), pages 93-104.
    5. Mathiesen, B.V. & Lund, H. & Connolly, D. & Wenzel, H. & Østergaard, P.A. & Möller, B. & Nielsen, S. & Ridjan, I. & Karnøe, P. & Sperling, K. & Hvelplund, F.K., 2015. "Smart Energy Systems for coherent 100% renewable energy and transport solutions," Applied Energy, Elsevier, vol. 145(C), pages 139-154.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yamashita, Daniela Yassuda & Vechiu, Ionel & Gaubert, Jean-Paul, 2020. "A review of hierarchical control for building microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 118(C).
    2. José Luis Ruiz Duarte & Neng Fan, 2022. "Operation of a Power Grid with Embedded Networked Microgrids and Onsite Renewable Technologies," Energies, MDPI, vol. 15(7), pages 1-24, March.
    3. Haochen Hua & Chuantong Hao & Yuchao Qin & Junwei Cao, 2018. "A Class of Control Strategies for Energy Internet Considering System Robustness and Operation Cost Optimization," Energies, MDPI, vol. 11(6), pages 1-20, June.
    4. Lund, Henrik & Thellufsen, Jakob Zinck & Sorknæs, Peter & Mathiesen, Brian Vad & Chang, Miguel & Madsen, Poul Thøis & Kany, Mikkel Strunge & Skov, Iva Ridjan, 2022. "Smart energy Denmark. A consistent and detailed strategy for a fully decarbonized society," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    5. Persson, Urban & Wiechers, Eva & Möller, Bernd & Werner, Sven, 2019. "Heat Roadmap Europe: Heat distribution costs," Energy, Elsevier, vol. 176(C), pages 604-622.
    6. Guelpa, Elisa & Bischi, Aldo & Verda, Vittorio & Chertkov, Michael & Lund, Henrik, 2019. "Towards future infrastructures for sustainable multi-energy systems: A review," Energy, Elsevier, vol. 184(C), pages 2-21.
    7. Wang, Yongli & Li, Jiapu & Wang, Shuo & Yang, Jiale & Qi, Chengyuan & Guo, Hongzhen & Liu, Ximei & Zhang, Hongqing, 2020. "Operational optimization of wastewater reuse integrated energy system," Energy, Elsevier, vol. 200(C).
    8. Besagni, Giorgio & Premoli Vilà, Lidia & Borgarello, Marco & Trabucchi, Andrea & Merlo, Marco & Rodeschini, Jacopo & Finazzi, Francesco, 2021. "Electrification pathways of the Italian residential sector under socio-demographic constrains: Looking towards 2040," Energy, Elsevier, vol. 217(C).
    9. Aleksandra Matuszewska-Janica & Dorota Żebrowska-Suchodolska & Urszula Ala-Karvia & Marta Hozer-Koćmiel, 2021. "Changes in Electricity Production from Renewable Energy Sources in the European Union Countries in 2005–2019," Energies, MDPI, vol. 14(19), pages 1-27, October.
    10. Iribarren, Diego & Martín-Gamboa, Mario & Navas-Anguita, Zaira & García-Gusano, Diego & Dufour, Javier, 2020. "Influence of climate change externalities on the sustainability-oriented prioritisation of prospective energy scenarios," Energy, Elsevier, vol. 196(C).
    11. Song, Kwonsik & Kim, Sooyoung & Park, Moonseo & Lee, Hyun-Soo, 2017. "Energy efficiency-based course timetabling for university buildings," Energy, Elsevier, vol. 139(C), pages 394-405.
    12. Terfa, H. & Baghli, L. & Bhandari, R., 2022. "Impact of renewable energy micro-power plants on power grids over Africa," Energy, Elsevier, vol. 238(PA).
    13. Chen, Yen-Haw & Lu, Su-Ying & Chang, Yung-Ruei & Lee, Ta-Tung & Hu, Ming-Che, 2013. "Economic analysis and optimal energy management models for microgrid systems: A case study in Taiwan," Applied Energy, Elsevier, vol. 103(C), pages 145-154.
    14. Felix Garcia-Torres & Ascension Zafra-Cabeza & Carlos Silva & Stephane Grieu & Tejaswinee Darure & Ana Estanqueiro, 2021. "Model Predictive Control for Microgrid Functionalities: Review and Future Challenges," Energies, MDPI, vol. 14(5), pages 1-26, February.
    15. Keon Baek & Woong Ko & Jinho Kim, 2019. "Optimal Scheduling of Distributed Energy Resources in Residential Building under the Demand Response Commitment Contract," Energies, MDPI, vol. 12(14), pages 1-19, July.
    16. Fridgen, Gilbert & Keller, Robert & Körner, Marc-Fabian & Schöpf, Michael, 2020. "A holistic view on sector coupling," Energy Policy, Elsevier, vol. 147(C).
    17. Li, Qiang & Gao, Mengkai & Lin, Houfei & Chen, Ziyu & Chen, Minyou, 2019. "MAS-based distributed control method for multi-microgrids with high-penetration renewable energy," Energy, Elsevier, vol. 171(C), pages 284-295.
    18. Svetlana Drobyazko & Suparna Wijaya & Pavel Blecharz & Sergii Bogachov & Milyausha Pinskaya, 2021. "Modeling of Prospects for the Development of Regional Renewable Energy," Energies, MDPI, vol. 14(8), pages 1-17, April.
    19. Amir Faraji & Maria Rashidi & Fatemeh Rezaei & Payam Rahnamayiezekavat, 2023. "A Meta-Synthesis Review of Occupant Comfort Assessment in Buildings (2002–2022)," Sustainability, MDPI, vol. 15(5), pages 1-36, February.
    20. Soha, Tamás & Munkácsy, Béla & Harmat, Ádám & Csontos, Csaba & Horváth, Gergely & Tamás, László & Csüllög, Gábor & Daróczi, Henriett & Sáfián, Fanni & Szabó, Mária, 2017. "GIS-based assessment of the opportunities for small-scale pumped hydro energy storage in middle-mountain areas focusing on artificial landscape features," Energy, Elsevier, vol. 141(C), pages 1363-1373.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:239:y:2019:i:c:p:598-609. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.