IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v289y2021ics0306261921002403.html
   My bibliography  Save this article

Surrogate model enabled deep reinforcement learning for hybrid energy community operation

Author

Listed:
  • Wang, Xiaodi
  • Liu, Youbo
  • Zhao, Junbo
  • Liu, Chang
  • Liu, Junyong
  • Yan, Jinyue

Abstract

Local peer-to-peer (P2P) transactions in a community are becoming a trend for energy integration and management. The introduction of P2P trading scheme requires comprehensive consideration on various aspects, such as peer privacy, computational efficiency, network security and operational economics. This paper provides a novel hybrid community P2P market framework for multi-energy systems, where a data-driven market surrogate model-enabled deep reinforcement learning (DRL) method is proposed to facilitate P2P transaction within technical constraints of the community delivery networks. Specifically, to achieve privacy protection, a market surrogate model based on deep belief network (DBN) is developed to characterize P2P transaction behaviors of peers in the community without disclosing their private data. Since the energy inputs and outputs of peers are highly correlated with real time signals of retail energy prices, the data-driven market surrogate model is further integrated into the DRL-enabled optimization model of a community agent (CA) for on-line retail energy price generation. Particularly, by integrating network constraints into DRL reward function, the P2P transaction scheme among community peers under specific retail energy price is guaranteed to proceed within a feasible region of community networks. Numerical results indicate that the proposed market framework can achieve 7.6% energy cost saving for community peers over none P2P transaction scheme while increase 284.4$ economic benefits for CA in one day over other comparison algorithms. This study provides an effective prototype to supplement existing P2P markets.

Suggested Citation

  • Wang, Xiaodi & Liu, Youbo & Zhao, Junbo & Liu, Chang & Liu, Junyong & Yan, Jinyue, 2021. "Surrogate model enabled deep reinforcement learning for hybrid energy community operation," Applied Energy, Elsevier, vol. 289(C).
    • Handle: RePEc:eee:appene:v:289:y:2021:i:c:s0306261921002403
    DOI: 10.1016/j.apenergy.2021.116722
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261921002403
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2021.116722?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Grigorios L. Kyriakopoulos, 2022. "Energy Communities Overview: Managerial Policies, Economic Aspects, Technologies, and Models," JRFM, MDPI, vol. 15(11), pages 1-45, November.
    2. Rehman, Anis Ur & Shafiq, Aqib & Ullah, Zia & Iqbal, Sheeraz & Hasanien, Hany M., 2023. "Implications of smart grid and customer involvement in energy management and economics," Energy, Elsevier, vol. 276(C).
    3. Yuan, Quan & Ye, Yujian & Tang, Yi & Liu, Yuanchang & Strbac, Goran, 2022. "A novel deep-learning based surrogate modeling of stochastic electric vehicle traffic user equilibrium in low-carbon electricity–transportation nexus," Applied Energy, Elsevier, vol. 315(C).
    4. Qiu, Dawei & Dong, Zihang & Zhang, Xi & Wang, Yi & Strbac, Goran, 2022. "Safe reinforcement learning for real-time automatic control in a smart energy-hub," Applied Energy, Elsevier, vol. 309(C).
    5. Lee, Minwoo & Han, Changho & Kwon, Soonbum & Kim, Yongchan, 2023. "Energy and cost savings through heat trading between two massive prosumers using solar and ground energy systems connected to district heating networks," Energy, Elsevier, vol. 284(C).
    6. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    7. Ghafariasl, Parviz & Mahmoudan, Alireza & Mohammadi, Mahmoud & Nazarparvar, Aria & Hoseinzadeh, Siamak & Fathali, Mani & Chang, Shing & Zeinalnezhad, Masoomeh & Garcia, Davide Astiaso, 2024. "Neural network-based surrogate modeling and optimization of a multigeneration system," Applied Energy, Elsevier, vol. 364(C).
    8. Gao, Datong & Zhao, Bin & Kwan, Trevor Hocksun & Hao, Yong & Pei, Gang, 2022. "The spatial and temporal mismatch phenomenon in solar space heating applications: status and solutions," Applied Energy, Elsevier, vol. 321(C).
    9. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    10. Chen, Yongdong & Liu, Youbo & Zhao, Junbo & Qiu, Gao & Yin, Hang & Li, Zhengbo, 2023. "Physical-assisted multi-agent graph reinforcement learning enabled fast voltage regulation for PV-rich active distribution network," Applied Energy, Elsevier, vol. 351(C).
    11. Zhu, Dafeng & Yang, Bo & Liu, Yuxiang & Wang, Zhaojian & Ma, Kai & Guan, Xinping, 2022. "Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park," Applied Energy, Elsevier, vol. 311(C).
    12. Alabi, Tobi Michael & Lu, Lin & Yang, Zaiyue, 2024. "Real-time automatic control of multi-energy system for smart district community: A coupling ensemble prediction model and safe deep reinforcement learning," Energy, Elsevier, vol. 304(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:289:y:2021:i:c:s0306261921002403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.